[clue-tech] Improving performance of php sending mail via exim on CentOS5

[Keith Hellman]

On Wed, Mar 03, 2010 at 04:56:36PM -0700, Greg Knaddison wrote:
> * When content is posted some code puts a list of mails to send into a
> database table which is then used as a queue. There is another that
> pulls e-mails out of the queue and actually sends them. The second one
> is running non-stop and currently processes mail at a rate of about
> 500 mails per minute.

By non-stop, you mean the process is always in the ps list, or that it
is consuming most or all of the processor?

> * This is a VPS and according to "cat /proc/cpuinfo | grep processor |
> wc -l" it thinks it has 8 cores.
> * When I run top the % of cpu for user processes rarely gets over 50%.
> mysqld occasionally consumes 100% or even 150% of the CPU, but my
> understanding of how top is reporting leads me to believe it is not

OK, the answer to my previous question is it is always in the list, but
not eating up processor cycles.  Yes?

> CPU bound. The %wa value is almost always 0, or occasionally a
> fraction of a percent so I think it's not disk IO bound.
> * I tried running "sar -n DEV -f /var/log/sa/sa24 | more" to get a
> sense of network traffic, but if I compare an hour when the mail
> processing script is running vs. not I don't see a big difference in
> any of the values.

Where are these values with respect to your network bandwidth?  I assume
they are not a large fraction of it.  7000 1K messages per minute
wouldn't be saturating a link, is there other a big chunk of other
network traffic?

Is there an upstream router that is gating your TCP connections?  Are
the processes themselves do this?  True story:  occasionally I have to
send out 100-200 Emails with (bigish) PDF attachments.  Naturally I have
a script that takes care of this quite nicely.

One day I get a flood of bounced messages --- the powers that be 
had installed a new mail server and it had decided that my notebook was
a compromised machine spewing forth spam.  (Despite the fact that each
message was gpg signed --- how many spammers do that?)  The solution was
simple:  sleep 2 between each message appears to be just below the mail
server threshold :^)

Of course, it runs 100x slower, and sometimes I forget about my little
2s pause, glance at the console window and wonder why this is taking so
long...  Is the code around this whole Email waltz something you (your
company) has written?  Or could it by chance be code that someone else
has artificially slowed down to make the blacklisting Gods happy?

Come to think of it, I'm sure this is not the root of your problems ---
but it was an excuse for a good (?) story.

-- 
Keith Hellman                             #include <disclaimer.h>
khellman at mcprogramming.com                from disclaimer import standard
khellman at mines.edu
                                   -*-                                    
                    public key @ pgp.mit.edu 9FCF40FD 
    Y!M: mcprogramming                           AIM/ICQ: 485403897       
   gtalk (xmpp jabber): mrtuple at jabber.org, jabber at mcprogramming.com                      
                                   -*-                                    

The purpose of computing is insight, not numbers.

-- Richard W. Hamming, 1962
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
Url : http://cluedenver.org/pipermail/clue-tech/attachments/20100303/f79d7410/attachment.bin