[clue-tech] 2.6 kernel compile options make Intel CPU slow

[Jim Ockers]

Hi,

Maxwell Spangler wrote:
> On Wed, 2010-08-04 at 14:31 -0600, Jim Ockers wrote:
>
>   
>> The problem is that when the cooling fails (failed fan or clogged heat
>> sink), we get a significant performance hit, so we need to know right
>> away if that happens and we don't always get good information about
>> the cooling.  The thing we see right away is slow system performance
>> but it manifests itself in a variety of weird and non-obvious ways.
>> We have determined that we need maximum performance and nothing less
>> is OK so we were trying to detect failures right away.
>>     
>>> So you can use a 2.4 kernel for cooling measurements.
>>>   
>>>       
>> A 2.6 kernel with SMP turned off behaves the same, scheduler-wise, as
>> the 2.4 kernel (and I'm sure that was not a SMP kernel either) on the
>> hyperthreading P4.
>>     
>>> Can you measure/detect CPU temperature (which ought to relate somehow to throttling)?
>>>   
>>>       
>> Yes but not reliably enough in our 2004-vintage hardware.
>>     
>>> Can you measure power consumption (which ought to decrease when throttling)?
>>>   
>>>       
>> Good idea, but unfortunately no in our old hardware.
>>     
>
> Is this proprietary hardware or some old 2004 era x86 PC motherboard?
>   
They are old Intel motherboards.
> If it's the latter I would think you could probe for the CPU speed and
> use that (assuming that high temperatures means lowered Mhz speed equals
> poor performance?).
>   
The Intel P4 CPU thermal throttling lowers the duty cycle of the CPU. No 
indication is given to the OS (including lowered reported CPU speeds) 
that this is happening. When we get a thermal failure the overall system 
starts failing in really weird ways but they do not at first glance 
point to thermal throttling.

The dd test was a quick & dirty way of telling if the CPU was on a 
lowered duty cycle. It may be that if we could use the "P4 thermal 
throttling interrupt" Linux kernel feature then that would expose this 
problem.
> I'm still really curious about your dd issue.
>
> dd if=/dev/zero of=/dev/null
>
> in my mind is
>
> 1) execute the dd command which simply reads in one channel and outputs
> to another
> 2) the input channel of /dev/zero should be so simple and fast that it
> can push a CPU at full speed (no i/o wait on calculating zeros, etc)
> 3) the output channel of /dev/null should be so simple and fast that it
> can push a CPU at full speed (no i/o wait on writes, etc)
>
> So why does a command that should be so simple take a performance hit in
> a hyperthreading environment?  It bothers me, although I'm sure not as
> much as you.
>
>   
Well trust me it bothers me a lot. I agree with you that it should not 
take a performance hit under HT. It doesn't seem to be affected by true 
SMP - there is no performance hit. We have now spent a lot of time 
trying to figure this out and aren't much closer. The nearest I can come 
to an explanation about the possibilities for why it's slow is this:

1. Linux kernel scheduler is not optimized for HT. Or, certain tasks 
have negative optimization, but maybe other tasks are positively optimized.
2. Architectural issue with Intel P4 HT, which makes some things slower 
but maybe makes other things faster.

Since we can disable HT in the BIOS we are going to run some overall 
system benchmark tests to try to figure out if the system as a whole 
(including all services & daemons, mysql, etc) is faster or slower with 
HT disabled or enabled.

Thanks for thinking about this everyone,
Jim

-- 
Jim Ockers, P.Eng. (ockers at ockers.net)
Contact info: http://www.ockers.ca/pason.html


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://cluedenver.org/pipermail/clue-tech/attachments/20100805/a79e62da/attachment.html