[CLUE-Tech] weird apache error?

[Sean Reifschneider]

>> Error: MM: mm:core: failed to acquire semaphore (No space left on
>> device): OS: Invalid argument
>
>Try "ipcs" to see whats up with the ipc structures.

Yeah, it seems to be related to apache leaking semaphores...  I was
getting:

   [error] (28)No space left on device: Cannot create SSLMutex file
   `/etc/httpd/logs/ssl_mutex.9850' Configuration Failed

It obviously wasn't out of file-system space...  "ipcs" showed that
there were around 50 semaphores allocated to Apache.  I went ahead and
nuked all those that were associated with Apache, then I was able to
re-start the server without any problem.  After the restart it only had
3 semaphores.

Sometimes they go away, sometimes they don't.  It's not happening all
the time either.  This is running on a KRUD 8.0 system with the latest
Apache errata applied as well.

It was annoying because sometimes it would fail to re-start properly
after the log rotation...  I've set up a couple of scripts that seem to
be resolving this problem -- one that forces a re-start at 5 minutes
after midnight, and another one that runs every minute and checks if
httpd is down.  If it is, apache-related semaphores are nuked and then
the server is re-started.  They're both pretty simple scripts, hopefully
this fully resolves the problem so I can stop getting paged randomly
just after midnight.

Sean
===================
This is a follow-up to a message on CLUE-Tech back in September -- since
I had additional information I figured I'd post it.
-- 
 I would rather suffer defeat than have cause to be ashamed of victory.
                 -- Quintus Curtius
Sean Reifschneider, Inimitably Superfluous <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995.  Qmail, Python, SysAdmin