| View previous topic :: View next topic |
| Author |
Message |
hardtarget
n00b
Joined: 05 Jan 2008
Posts: 15
Location: Seattle, WA
|
 Posted: Thu May 08, 2008 12:31 am Post subject: Slowing transfer speed for copying files to a NFS share. |
 |
|
Hello,
First a little history. I have a NFS server (Dual Pentium 3, 1.5GB RAM, 1 x Intel Gigabit NIC) connected to another file server (Dual 2.8Ghz Xeon, 1GB RAM, 2 x Intel Gigabit NICs) via a crossover cable, with Jumbo Frames enabled (MTU 9000). The other NIC in the file server is connected to a 100Mb Switch, standard MTU. The Xeon fileserver is Gentoo, and the NFS server is FreeBSD 6.2.
NFS Server Exported Share = /mnt/shared
File Server NFS Mounted Share = /home/shared
When I SCP a file from a desktop machine (also running Gentoo, and connected to the same switch as the file server) to /home/shared on the file server, I get an initial speed of ~10MB/sec, but it then quickly (within about 10-15 seconds) steadily drops off to 8MB, 6MB, 4MB, 2MB, 600KB, and then eventually stalls. The actual mounted directory (/home/shared) stays "alive", and I can repro this every time I SCP (or even FTP) a file to the NFS mounted directory (/home/shared) on the file server. If I change the MTU from 9000 to 1500 on both ends of the links between the NFS server and file server, I see no difference in behaviour.
The Xeon File Server is using the following options to mount the remote NFS share:
rw,nfsvers=3,rsize=8192,wsize=8192,timeo=14,intr,async,hard
Has anybody seen anything like this before? I would greatly appreciate any assistance. I've done several tcpdumps and don't see any excessive packet fragmentation happening, specific network slowdowns (or anything else that looks abnormal) on the 100MB or 1GB links. |
|
| Back to top |
|
 |
pianosaurus
l33t
Joined: 19 Apr 2004
Posts: 943
Location: Bash$
|
| Posted: Thu May 08, 2008 4:59 am Post subject: |
|
|
Hmm. Puzzling. Have you made sure the drop-off is not actually abrupt? If you cut the link somehow, most transfer apps will show a slowly decreasing transfer speed, just because they do a regular averaging calculation.
Also strange that it happens in both scp and ftp. Can you copy a big file to /home/shared from e.g. a CD? It could be a problem with all writing to that partition.
_________________
PKA Cuber
Please add [SOLVED] to the subject of your original post when you feel that your problem is resolved.
Adopt an unanswered post |
|
| Back to top |
|
|
hardtarget
n00b
Joined: 05 Jan 2008
Posts: 15
Location: Seattle, WA
|
| Posted: Thu May 08, 2008 5:37 am Post subject: |
|
|
| Cuber wrote: | | Can you copy a big file to /home/shared from e.g. a CD? It could be a problem with all writing to that partition. |
Thanks for the reply. I've tried a local copy on the File Server (just using "cp /path/to/bigfile.tgz /home/shared/"), and that works fine. It's only when SCP'ing or FTP'ing (or any other type of network transfer) to the file server's mounted NFS share that the problem happens.
I don't think it's anything hardware related either, since I can copy via SCP or FTP to another account on the file server, as long as it isn't to /home/shared. |
|
| Back to top |
|
|
hardtarget
n00b
Joined: 05 Jan 2008
Posts: 15
Location: Seattle, WA
|
| Posted: Thu May 08, 2008 11:31 pm Post subject: |
|
|
| hardtarget wrote: | | I've tried a local copy on the File Server (just using "cp /path/to/bigfile.tgz /home/shared/"), and that works fine. |
I retract that statement above. I did some additional testing, and noticed even with a "cp /path/to/bigfile.tgz /home/shared/", I am getting at most ~15Mb across the gigabit link. To me, that sounds entirely too slow. Does anybody know some common tunables I can try to help improve the performance? |
|
| Back to top |
|
|
hardtarget
n00b
Joined: 05 Jan 2008
Posts: 15
Location: Seattle, WA
|
| Posted: Sun May 11, 2008 1:23 am Post subject: |
|
|
*bump*
Sorry to bump this thread again, but I am at a loss here. Any NFS gurus out there willing to assist? |
|
| Back to top |
|
|
SeaTiger
l33t
Joined: 22 Nov 2007
Posts: 603
Location: Toronto, Ontario, Canada
|
| Posted: Tue May 13, 2008 12:43 am Post subject: |
|
|
try | Code: | | rw,nfsvers=3,rsize=32768,wsize=32768,timeo=14,intr,async,hard |
Did you test copying from fileserver (instead of from a desktop machine) into the share folder? Problem may be between fileserver and nfs server link. |
|
| Back to top |
|
|
hardtarget
n00b
Joined: 05 Jan 2008
Posts: 15
Location: Seattle, WA
|
| Posted: Tue May 13, 2008 3:57 pm Post subject: |
|
|
For those that are interested, I was able to resolve this by increasing (by 10 times) the number of NFSd servers that get started on the NFS Server. It's now @ 40 NFS servers, and I can run about 75Mb consistently between the Linux fileserver and the NFS server, with spikes to 120-130Mb.
My /etc/fstab looks like this for my NFS mounts on the Linux fileserver:
| Code: | 10.99.18.254:/mnt/vmware /home/vmware nfs rw,nfsvers=3,rsize=32768,wsize=32768,timeo=10,intr,hard,udp,async 0 0
10.99.18.254:/mnt/shared /home/shared nfs rw,nfsvers=3,rsize=32768,wsize=32768,timeo=10,intr,hard,udp,async 0 0 |
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
