[clue-tech] Improving performance of php sending mail via exim on CentOS5

[Greg Knaddison]

On Wed, Mar 3, 2010 at 8:46 PM, Keith Hellman
<khellman at mcprogramming.com> wrote:
> By non-stop, you mean the process is always in the ps list, or that it
> is consuming most or all of the processor?

Correct, it's always there. Technically it's an infinite loop in bash
that calls the php code to send mails for about an hour and then it
sleeps for 5 seconds.

It's something we should turn into a daemon as things get more stable,
but for now the shell script is working ok.

>
> OK, the answer to my previous question is it is always in the list, but
> not eating up processor cycles.  Yes?

It's eating them - about 10-30% of the CPU at varying points - but
never really maxing it out.

> Where are these values with respect to your network bandwidth?  I assume
> they are not a large fraction of it.  7000 1K messages per minute
> wouldn't be saturating a link, is there other a big chunk of other
> network traffic?

The messages are all plain text and relatively small, but I have no
idea if they are saturating the link. How could I monitor that?

> Is there an upstream router that is gating your TCP connections?  Are
> the processes themselves do this?  True story:  occasionally I have to
> send out 100-200 Emails with (bigish) PDF attachments.  Naturally I have
> a script that takes care of this quite nicely.
>
> One day I get a flood of bounced messages --- the powers that be
> had installed a new mail server and it had decided that my notebook was
> a compromised machine spewing forth spam.  (Despite the fact that each
> message was gpg signed --- how many spammers do that?)  The solution was
> simple:  sleep 2 between each message appears to be just below the mail
> server threshold :^)
>
> Of course, it runs 100x slower, and sometimes I forget about my little
> 2s pause, glance at the console window and wonder why this is taking so
> long...  Is the code around this whole Email waltz something you (your
> company) has written?  Or could it by chance be code that someone else
> has artificially slowed down to make the blacklisting Gods happy?
>
> Come to think of it, I'm sure this is not the root of your problems ---
> but it was an excuse for a good (?) story.

It's open source/Drupal code and we've written some of it, so I'm
fairly certain that there's nothing artificially slowing it. The
hosting company does have a hard limit on mails-per-hour but this is
something we can control and we've set it to 120,000 per hour.

And yes, that is a good story!

Cheers,
Greg


-- 
Greg Knaddison | 303-800-5623 | http://growingventuresolutions.com
Mastering Drupal - http://www.masteringdrupal.com