[clue-tech] CLUE Talk Mailing list mbox file too big to rsynch

[Jed S. Baer]

On Mon, 2 Feb 2009 16:52:32 -0700
Kevin Fenzi wrote:

> > I'll see what I can find to get more logging going on the server side.
> > Turning up the verbosity on the issuing end hasn't turned up anything.
> 
> It does seem like an issue on the server end... 
> 
> Check for any OOM messages there? Perhaps something is taking up a
> bunch of memory and the rsync doesn't have enough to gather a
> filelist... 

I think that's the nub of it, though it's still perplexing to me why
it would happen only on the CLUE Talk directory tree, and not the larger
CLUE-Tech load. However, further digging around does reveal that other
processes are dying periodically, presumably due to lack of memory, or in
the case of Postfix, definitely due to lack of memory.

I'm seeing occasional dovecot IMAP messages reporting a kill 9. Not sure
that this is the kernel's OOM killer, because I hadn't heard of that
before, but just ran across the term here:

http://forum.slicehost.com/comments.php?DiscussionID=2799&page=1

However, I'm not finding things in /var/log/messages with the sort of
detail reported on that page -- in fact, I'm finding damn little in
messages.

Given the memory contraints in our VPS hosting package, being well and
truly OOM seems the most likely culprit. I might even infer that there's
not a lot I can do about it. But I can try to track down peak load times,
and schedule around them. Really, I thought 2AM would be safe, but maybe
that's when the spambots unleash their vile effluent, in which case
Postfix would be busy tossing it all in the trash, or delivering it to
Mailmain to be tossed in the trash. I will note that on Sunday, I tried
it at various times during the day, still getting the error. However,
early this AM, it apparently ran with no errors (I get e-mail from cron
if there are errors).

Not to poo-poo Angelo's suggestion of splitting the Mailman archives, but
I'm not sure that's readily do-able. Or maybe it is, and I can't find the
instructions.

jed