[clue-tech] NFS/server tuning with mail-ldap and maildirs

[Greg Knaddison]

On 6/17/05, Adam bultman <adamb at glaven.org> wrote:
> 
> Roy J. Tellason wrote:
> 
> >On Thursday 16 June 2005 03:42 pm, Adam bultman wrote:
> >
> >
> >>Greetings.
> >>
> >>A few questions for you cluesters out there:
> >>
> >>I'm working on getting qmail-ldap, courier-imap/pop3 on serverblades, with
> >>NFS mounted maildirs. However, when mailboxes get larger than a few
> >>thousand, things slow down a great deal, more than they should for a system
> >>like that.
> >>
> >>The server will either spend time with imapd at about 10% usage, with it
> >>waiting on the network, or imapd will take up ALL the cpu, and spin for
> >>quite some time.
> >>
> >>My configuration is this:
> >>
> >>Netapp, with gigabit connection to the switch.
> >>
> >>Server: 800 MHz serverblade, 512 MB RAM, 100Mbit connection to switch.
> >>
> >>
> >
> ><...>
> >
> >
> >
> >>Are there more effective NFS options for these servers to use?  Right
> >>now, I'm using soft, noatime, tcp as my options. I don't know if using
> >>TCP is any *faster*, or if setting the rsize or wsize will help any either.
> >>
> >>
> >
> ><...>
> >
> >
> >
> >>Any ideas would be most appreciated.  I'm wondering if there's a
> >>fundamnetal speed problem that I won't be able to overcome with more
> >>serverblades, or if there's other ways I could squeak more performance out
> >>of them.
> >>
> >>
> >
> >I too am seeing a serious performance hit with having moved my mail setup from
> >all-in-one-box to using nfs,  though I'm not using anywhere near the hardware
> >capabilities you're talking about here,  just a small home LAN setup.  I've
> >found that keeping stuff with fairly short expiration times (and therefore a
> >smaller number of messages overall) seems to help,  but aside from that I'm
> >not sure what to try next either.
> >
> >I will be looking at other replies to your post with some interest.
> >
> >
> >
> >
> >
> >
> Well, I've been doing more research, and finding that people are usually
> using slower machines for their smtp servers, and putting more muscle
> behind their imap/pop servers. At any given time, the SMTP server might
> be procssing a few dozen messages, maybe a hundred on a busy box, but
> imap will pretty much be constant, and high-demand.
> 
> I'll be testing some of it in the upcoming weeks, and we'll see how
> things work.  I'm expecting to find that the slower systems (800 Mhz
> blades) are more than acceptable for SMTP work, but not quite cut out
> for the imap/pop3 work.
> 
> I'm thinking that for more responsive imap/pop3, I'll need more RAM, a
> faster CPU, and ideally, a gigabit connection to reduce latency on the
> network (hopefully crossover)
> 

I've never seen a network mount run quickly enough to satisfy people
who were in "performance" situations.  I haven't seen a lot of people
needing to do it, but it is just a bit flaky in my experience and
stories from trusted coworkers.  Since IMAP/POP is your bottleneck,
what about attaching the storage directly to that box and NFS mounting
it from the SMTP server?

That way the people who care about performance (IMAP/POP) get the fast
disk access and the messages that are already asynchronous operations
(SMTP) can be a little more asynchronous.

Greg