[CLUE-Tech] Network DD

[Michael Robbert]

Tim,
Are you taking into account the amount of time it takes to read the data 
from disk. I don't have any suggestions off the top of my head, but I 
think that you'll want to make sure that all your data is loaded in 
memory before you start timing the data transfer. Also, are you sure 
that there is no other traffic on your network. If you're grabbing data 
from ifconfig it'll be skewed by any other traffic that might hit the 
interface.

Timothy C. Klein wrote:
> * Keith Hellman (kehellman at yahoo.com) wrote:
> 
>>I'm wondering if your accurately counting the actual bytes going across
>>the interface.  Your file may be X bytes big but your byte calculation
>>doesn't account for the tcp wrapper, ip wrapper, or link layer envelope.
>>
>>I'm not even sure where to get these numbers, but I would suggest doing a 
>>  >/sbin/ifconfig ethX
>>  >dd if=/dev/random bs=4 count=4 |netcat target.ip.address target-port
>>  >/sbin/ifconfig ethX
>>
>>And compare the total transferred bytes between the two ifconfig outputs. 
>>I think they should be more than 16 bytes...
> 
> 
> Thanks, I had forgot about that.  It adds some inaccuracy, becuase I
> don't want to eliminate all data transfer between the two, but it is
> still interesting.  The other overhead, though, does not seem to be
> really significant.  Just treid 4 runs with netcat, I get pretty much
> the same result.  Right about 30% utilization of the 100 Mb/s. That is
> when I measure the difference in bytes between ifconfig outputs, rather
> than the file size.  I realize that I may never get all the way to 100%,
> but less than 1/3 is bothering me.  I must find out why.  A new
> project...
> 
> 
>>Also keep in mind, IIRC, netcat is doing VERY dumb byte transfers: give it
>>a byte, it sends a byte.  The SSH tools, on the other hand, may (and this
>>is pure speculation) be more optimized (especially scp) to moving large
>>blocks of data, hence they do it with less protocol overhead, hence
>>faster.
> 
> 
> Ah, but I want dumb transfers.  I just want to measure how long a known
> set of data takes to transfer.  I don't want compression, tons of tcp
> overhead, etc.
> 
> 
>>Someone knowledgeable about this stuff should comment, I'm really just
>>thinking out loud...
>> 
> 
> 
> Gonna have to test some more.  These machines are separted by 25' of
> Cat5 wire that may not be up to 100Mb/s spec, so I am going to try it
> with a short piece of Cat 5e wire, just to rule out a lame line.
> 
> I have always had the feeling that the stock Linux network drivers seem
> to behave *very* conservatively 'out of the box', and I am going to find
> out for sure!
> 
> Tim
> --
> ==============================================
> == Timothy Klein || teece at silverklein.net   ==
> == ---------------------------------------- ==
> == "Hello, World" 17 Errors, 31 Warnings... ==
> ==============================================
> _______________________________________________
> CLUE-Tech mailing list
> CLUE-Tech at clue.denver.co.us
> http://clue.denver.co.us/mailman/listinfo/clue-tech


-- 
Michael "Murph" Robbert
System Administrator for Math/CS
Colorado School of Mines, Golden, CO  80401-1887
Office: SH220
Office phone: 303-273-3786
Pager: 303-461-6543 or Text messages: murph_pager at bigfoot.com
Email: mrobbert at mines.edu