View previous topic :: View next topic |
Author |
Message |
Alan Turing n00b
Joined: 10 Sep 2003 Posts: 53 Location: Freiburg, Baden (Germany)
|
Posted: Wed Mar 12, 2014 2:50 am Post subject: NFS4 much slower than NFS3 (when using subversion) |
|
|
I have a file server (using kernel 3.11.7-hardened-r1) on which an nfs server is running.
I have a desktop computer (using kernel 3.10.25-gentoo) which is mounting some folders which are exported by the server.
I am using subversion, the repository is somewhere in the internet, my local checkout is on the fileserver. This local checkout is also exported via nfs and sometimes mounted by the client.
Unfortunately this is very slow. An "svn up" done on the client takes around 135s. If I export the local checkout via samba, an "svn up" takes only around 20s.
Annoying: The "svn up" on the nfs filesystem seems to be slow for no reason. Neither on the client nor on the server I can see high activity with top or iotop.
However, mounting a filesystem via samba has some other disadvantages, and I live with this slow-svn-over-nfs problem for a few years now.
A few month ago, I switched from nfs3 to nfs4 and guess what? This made it even slower.
Meanwhile, I was too annoyed and made the following benchmarks.
First, I copied a 900MB file from the client to the nfs share.
With both, nfs3 and nfs4 this took around 9s. 100MB/s (over gigabit ethernet) - nothing to complain!
Furthermore, I copied the clients /usr/portage/dev-* (33102 files, together 100MB) to the nfs share.
With nfs3 this took 95s with nfs4 this took 135s.
Finally, I also compared the "svn up" on my local checkout. (For the case where there are no changes in the repository)
With nfs3 this took 135s with nfs4 this took 260s.
The /etc/exports on my server contains the line
Code: | /storage 192.168.0.42(rw,async,no_root_squash,no_subtree_check) |
The /etc/fstab on my client contains the line
Code: | 192.168.0.23:/storage /mnt/nfs nfs noauto,nfsvers=4,retry=0,noatime,nocto,nolock,local_lock=all 0 0 |
I also tried different mount options and different export options. Using other options, the performance was slightly worse but there was always a big difference between nfs3 and nfs4.
Do you have any comments or suggestions for me? |
|
Back to top |
|
|
666threesixes666 Veteran
Joined: 31 May 2011 Posts: 1248 Location: 42.68n 85.41w
|
Posted: Wed Mar 12, 2014 4:57 am Post subject: |
|
|
"Do you have any comments or suggestions for me?" suggestion... svn2git & git.... sftp & sshfs-fuse.... nfs, out the window, svn out the window... there are many ways to do things, and you're giving your self more means to the same ends.....
Last edited by 666threesixes666 on Wed Mar 12, 2014 9:55 am; edited 1 time in total |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2004
|
Posted: Wed Mar 12, 2014 9:23 am Post subject: |
|
|
I'm in no way an NFS or SVN expert, but I've noticed that NFS can run slowly if you don't have the correct setup for handling ID mapping; I presume it's using some slow work-around in such circumstances. Hope that helps. _________________ Greybeard |
|
Back to top |
|
|
Alan Turing n00b
Joined: 10 Sep 2003 Posts: 53 Location: Freiburg, Baden (Germany)
|
Posted: Wed Mar 12, 2014 9:02 pm Post subject: |
|
|
I tested the sshfs fuse filesystem (using its default options)
Copying the 900MB file took 30s.
Three times as much as nfs3. But both cores of the servers CPU where at 100%.
Copying the 33102 files from /usr/portage/dev-* took 14m9s
Nine times as much as nfs3. But no CPU was used more than 10%.
Doing my "svn up" took 55s.
Faster than nfs3 !
Using git to access the svn repository?
That is an option than I am going to check. Might be complicated, because I want to use a client that is integrated in the eclipse IDE.
Problem with ID mapping?
Sounds reasonable, since with nfs4 there came this idmapd daemon. I am going to check this. |
|
Back to top |
|
|
krinn Watchman
Joined: 02 May 2003 Posts: 7470
|
Posted: Thu Mar 13, 2014 5:17 am Post subject: |
|
|
nfsv4 define a root in exports, if it's not there, you aren't doing it right. I'm not sure if newer implementations has simply drop this or not, but older server was accepting it , the v4 standard explicit the need of root, the implementation without one was out of the standard and all distros was doing what they want. Result was full of bugs. |
|
Back to top |
|
|
666threesixes666 Veteran
Joined: 31 May 2011 Posts: 1248 Location: 42.68n 85.41w
|
Posted: Thu Mar 13, 2014 6:00 am Post subject: |
|
|
svn2git, as in exporting your repo to git. then accessing git via git
Quote: | dev-vcs/svn2git
Tool for one-time conversion from svn to git. |
so was sshfs fuse a good thing? im just throwing ideas out there. ssh is encrypted so certainly more secure than nfs. git by default uses ssh to push and pull commits.
https://www.eclipse.org/egit/
i guess its supported via egit? i dont use eclipse so its just a possibility being stated as a possibility.
its all moot to me i haven't looked into any bench marked comparisons.
gentoo's git wiki's starting to get pretty saucy... https://wiki.gentoo.org/wiki/Git
i like git because linus made it, and it revolves around diffs and is ultimately a patching system. i like git because it doesnt require svn repo upstream, you can git a folder on your desktop up and that is the upstream... git flow of git 2 git instead of client svn-srv |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21624
|
Posted: Fri Mar 14, 2014 1:41 am Post subject: |
|
|
For converting a Subversion repository to Git, I prefer using git svn, which ships with core Git and is available with USE=subversion. However, OP may be happy with his legacy centralized VCS and prefer to solve the NFS slowness. Older Subversion was very hard on inodes due to how the shadow information was stored. Newer versions seem to be better, but may still be too heavy for use over NFS. It would be useful to know why Subversion performs so poorly here. OP, could you post a breakdown of number of each type of NFS request issued? |
|
Back to top |
|
|
Alan Turing n00b
Joined: 10 Sep 2003 Posts: 53 Location: Freiburg, Baden (Germany)
|
Posted: Sun Mar 16, 2014 3:43 pm Post subject: |
|
|
I also defined a root in /etc/exports. I think the only difference is that you add fsid=0 to your root export in /etc/exports and use paths relative to this root in your mount options on the client.
So far I never noticed a difference between exporting a root via nfs or defining the exports in the old nfs3 style.
If the sshfs for me a good thing? I am indecisive. As written above, it was not always faster but it was also not always slower.
Converting the repository to GIT is not an option for me. I am neither the admin nor the only user of the repository.
I tried to obtain the NFS requests using nfsstat. Therefore, I rebootet the client to reset the counting of requests, did one "svn up" directly afterwards, and executed "nfsstat -l"
Using the following line in /etc/exports on the server
Code: | /storage 192.168.0.42(fsid=0,rw,async,no_root_squash,no_subtree_check) |
and the following line in /etc/exports on the client
Code: | 192.168.0.23:/ /mnt/nfs nfs noauto,nfsvers=4,retry=0,noatime,nocto,nolock,local_lock=all 0 0 |
I obtained the following output
Code: |
bionseY ~ # nfsstat -l
nfs v4 client total: 47171
------------- ------------- --------
nfs v4 client read: 16551
nfs v4 client write: 29
nfs v4 client commit: 2
nfs v4 client open: 17
nfs v4 client open_conf: 1
nfs v4 client close: 8
nfs v4 client fsinfo: 5
nfs v4 client renew: 3
nfs v4 client setclntid: 3
nfs v4 client confirm: 3
nfs v4 client lock: 6594
nfs v4 client locku: 6573
nfs v4 client access: 3272
nfs v4 client getattr: 3395
nfs v4 client lookup: 3260
nfs v4 client lookup_root: 2
nfs v4 client remove: 7
nfs v4 client pathconf: 3
nfs v4 client statfs: 8
nfs v4 client readdir: 4144
nfs v4 client server_caps: 8
nfs v4 client rel_lkowner: 3283
|
Afterwards I did the same with nfs3 and obtained the following output.
Code: |
bionseY ~ # nfsstat -l
nfs v3 client total: 30318
------------- ------------- --------
nfs v3 client getattr: 3321
nfs v3 client lookup: 3256
nfs v3 client access: 3260
nfs v3 client read: 16561
nfs v3 client write: 27
nfs v3 client create: 7
nfs v3 client remove: 7
nfs v3 client readdirplus: 3870
nfs v3 client fsstat: 4
nfs v3 client fsinfo: 2
nfs v3 client pathconf: 1
nfs v3 client commit: 2
|
|
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|