NFS File copies mysteriously slow down
2007-12-24 21:50:00
server ('servera') with 3 A1000s to a Solaris 8 server ('serverb') with a
Compaq storage array. I am doing this by NFS mounting the file systems from
servera on serverb over a gigabit ethernet link -- both servers plugged into
the same switch.
I start up several cpio commands to copy the files from the nfs mounted
filesystems to their destination on the Compaq array. Initially "netstat -i
60" shows over 300,000 packets per minute going across the wire (there is no
other network activity, just the nfs traffic and my ssh sessions to run the
cpio commands). I left it to run overnight, and this morning "netstat -i
60" is showing 30,000 - 40,000 packets per minute -- a 90% decrease. (Also,
iostat -x 15 on servera showed anywhere from 6,000 - 9,000 kr/s yesterday,
and this morning shows about 600 - 700 kr/s.) None of the cpio commands I
started have finished. They have not stalled either, but they are going
very slow now.
Both servers have load averages below 0.10 (both have 4 CPUs), and top shows
CPU as > 90% idle. iostat -x on both servers show %b < 10 for all devices
*except* for the following on serverb:
extended device statistics
device r/s w/s kr/s kw/s wait actv svc_t %w %b
nfs1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
nfs2 0.9 0.0 23.5 0.0 0.0 0.9 942.1 0 46
nfs3 2.2 0.0 67.6 0.0 0.0 1.4 619.1 0 64
nfs4 18.7 0.0 559.9 0.0 8.6 12.4 1128.2 94 100
So nfs appears to be holding things up (100%b, svc_t over 1000, wait is
8.6), but why? nfs does not appear to be fully utilizing CPU, disk, or
network, so what is slowing it down? Is there anything I can do to get this
back up to the speeds I was seeing when it started?
Will summarize, thanks in advance.
Doug
Comments
Got something to say?
You must be logged in to post a comment.
