[clue] [tech] Calculate total bandwidth usage for an interface using rrdtool

[Jim Ockers]

Hi CLUEbies,

Well as you know I send an e-mail to this list whenever I'm kind of 
stumped.  I have an OpenWRT based Linux system with a 3G phone 
interface.  Due to the marketecture of 3G data plans, I need to try to 
make sure this device does not exceed its monthly bandwidth allotment 
per our data plan, so we don't get a $60,000 bill, or something.  What I 
was thinking was, since we are allowed 5GB in a month, that I would 
figure out the total bandwidth used in the last 3 weeks, and if the 
total bandwidth was 3/4 * 5GB then I would start throttling down the 
available bandwidth on the interface using tc and traffic shaping.

This seems like it should not be too hard to do.

We have rrdtool and a "rra" datafile for the interface.  The byte 
counters are written from the interface into the RRA at more or less 
regular intervals, and rrdtool stores them as a time series rate.  For 
those of you new to rrdtool, the COUNTER datatype in a rra means that 
absolute values are converted to a rate upon insertion into the database.

It SEEMS like it should be really easy to get rrdtool to calculate and 
show me the average value of the bandwidth usage for the last 3 weeks.  
After all, AVERAGE is one of the command line options to rrdtool fetch, 
and you can specify a time range for the fetch/query.

Unfortunately, there seems to be no way to get rrdtool to actually do 
the math on this for me.  It returns a lot of rows of data with 
constant-interval timestamps, and each data value is the average and 
maximum value for (or around - it interpolates) each time stamp.  That's 
not what I want.  What I want is ONE result, at the current time, with 
the AVERAGE bandwidth usage for the amount of time I specified on the 
command line.  NaNs should be treated as zero (in the event of missing 
data because it was powered off or something), since presumably no data 
is being transferred over the 3G cell network while it's off.  However 
rrdtool refuses to do math on NaNs, and it also doesn't seem like it 
wants to give me time intervals greater than 44.5 minutes.  I'd like a 3 
week time interval, and I want it to interpolate and use zero for NaNs, 
and give me the AVERAGE BANDWIDTH USAGE FOR THE LAST 3 WEEKS IN ONE NUMBER.

Does anyone have any ideas?  Believe it or not an entire morning of 
google searching was unhelpful.  There are other people who want to do 
something like this and it seems they wind up using another Big Fancy 
Tool, or taking the time to write a C program.  Maybe rrdtool is the 
wrong tool for this job, even though on the surface it seems like the 
perfect tool for this job.

Thanks for any CLUEs,
Jim

-- 
Jim Ockers, P.Eng. (ockers at ockers.net)
Contact info: http://www.ockers.net/msi.html


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://cluedenver.org/pipermail/clue/attachments/20110610/5dc7dce0/attachment.html