NFS performance problem
2007-12-24 22:14:00
I'm running a large-ish mail system: 250,000 inbound emails per day,
270,000 POP3 accesses per day. It's outgrown its quad CPU E450 and is now
arranged between two similarly configured E450s.
The POP3 server is running Solaris 8, inbound server Solaris 2.6. I'm
running NFS on the POP3, where the mail is stored. It arrives on the
inbound server, auths, then gets delivered over NFS to the store, which is
in maildir format.
The problem I'm having is this: I need to find the size of a user's mailbox
as part of the authentication process. This involves stat()ing every file,
which works, but sometimes it seems that NFS simply can't respond quick
enough and eventually all my sendmail processes get stuck running the
external auth script, which takes forever. This seems to happen when I
have > 40 deliveries taking place simultaneously.
If I remove the stat() checks, the whole thing runs fine, but I may be
allowing users more than their allocated storage. In desperation, I tried
calling du -sk from my perl code. This was similarly unsuccessful.
I clearly need some NFS tuning, but I am unaware of exactly. On the client,
I've done this:
set nfs:nfs_nra=10
set nfs:nfs3_nra=10
set nfs:nfs_max_threads=128
set nfs:nfs3_max_threads=128
And I'm running the server like this:
root 10464 1 0 23:52:16 ? 0:00 /usr/lib/nfs/mountd
daemon 9292 1 0 23:51:08 ? 0:00 /usr/lib/nfs/statd
root 10466 1 0 23:52:16 ? 0:00 /usr/lib/nfs/nfsd -a 2048
root 9291 1 0 23:51:08 ? 0:00 /usr/lib/nfs/lockd
.. just in case I was running out of threads. It ran just as poorly with
the default number of threads. I have done no kernel tuning as such on the
server. Both machines are on a switched 100Mb LAN.
I will summarise.
Cheers
John P Connor
ICL.net
Comments
Got something to say?
You must be logged in to post a comment.
