NFS4 much slower than NFS3 (when using subversion)

Alan Turing

I have a file server (using kernel 3.11.7-hardened-r1) on which an nfs server is running.
I have a desktop computer (using kernel 3.10.25-gentoo) which is mounting some folders which are exported by the server.

I am using subversion, the repository is somewhere in the internet, my local checkout is on the fileserver. This local checkout is also exported via nfs and sometimes mounted by the client.

Unfortunately this is very slow. An "svn up" done on the client takes around 135s. If I export the local checkout via samba, an "svn up" takes only around 20s.
Annoying: The "svn up" on the nfs filesystem seems to be slow for no reason. Neither on the client nor on the server I can see high activity with top or iotop.

However, mounting a filesystem via samba has some other disadvantages, and I live with this slow-svn-over-nfs problem for a few years now.

A few month ago, I switched from nfs3 to nfs4 and guess what? This made it even slower.
Meanwhile, I was too annoyed and made the following benchmarks.

First, I copied a 900MB file from the client to the nfs share.
With both, nfs3 and nfs4 this took around 9s. 100MB/s (over gigabit ethernet) - nothing to complain!

Furthermore, I copied the clients /usr/portage/dev-* (33102 files, together 100MB) to the nfs share.
With nfs3 this took 95s with nfs4 this took 135s.

Finally, I also compared the "svn up" on my local checkout. (For the case where there are no changes in the repository)
With nfs3 this took 135s with nfs4 this took 260s.

The /etc/exports on my server contains the line

666threesixes666 · Posted: Wed Mar 12, 2014 4:57 am Post subject:

"Do you have any comments or suggestions for me?" suggestion... svn2git & git.... sftp & sshfs-fuse.... nfs, out the window, svn out the window... there are many ways to do things, and you're giving your self more means to the same ends.....

Goverp · Veteran Joined: 07 Mar 2007 Posts: 1994

I'm in no way an NFS or SVN expert, but I've noticed that NFS can run slowly if you don't have the correct setup for handling ID mapping; I presume it's using some slow work-around in such circumstances. Hope that helps.
_________________
Greybeard

Alan Turing · Posted: Wed Mar 12, 2014 9:02 pm Post subject:

I tested the sshfs fuse filesystem (using its default options)

Copying the 900MB file took 30s.
Three times as much as nfs3. But both cores of the servers CPU where at 100%.

Copying the 33102 files from /usr/portage/dev-* took 14m9s
Nine times as much as nfs3. But no CPU was used more than 10%.

Doing my "svn up" took 55s.
Faster than nfs3 !

Using git to access the svn repository?
That is an option than I am going to check. Might be complicated, because I want to use a client that is integrated in the eclipse IDE.

Problem with ID mapping?
Sounds reasonable, since with nfs4 there came this idmapd daemon. I am going to check this.

krinn · Watchman Joined: 02 May 2003 Posts: 7470

nfsv4 define a root in exports, if it's not there, you aren't doing it right. I'm not sure if newer implementations has simply drop this or not, but older server was accepting it , the v4 standard explicit the need of root, the implementation without one was out of the standard and all distros was doing what they want. Result was full of bugs.

666threesixes666 · Posted: Thu Mar 13, 2014 6:00 am Post subject:

svn2git, as in exporting your repo to git. then accessing git via git

Hu · Moderator Joined: 06 Mar 2007 Posts: 21595

For converting a Subversion repository to Git, I prefer using git svn, which ships with core Git and is available with USE=subversion. However, OP may be happy with his legacy centralized VCS and prefer to solve the NFS slowness. Older Subversion was very hard on inodes due to how the shadow information was stored. Newer versions seem to be better, but may still be too heavy for use over NFS. It would be useful to know why Subversion performs so poorly here. OP, could you post a breakdown of number of each type of NFS request issued?

Alan Turing · Posted: Sun Mar 16, 2014 3:43 pm Post subject:

I also defined a root in /etc/exports. I think the only difference is that you add fsid=0 to your root export in /etc/exports and use paths relative to this root in your mount options on the client.
So far I never noticed a difference between exporting a root via nfs or defining the exports in the old nfs3 style.

If the sshfs for me a good thing? I am indecisive. As written above, it was not always faster but it was also not always slower.

Converting the repository to GIT is not an option for me. I am neither the admin nor the only user of the repository.

I tried to obtain the NFS requests using nfsstat. Therefore, I rebootet the client to reset the counting of requests, did one "svn up" directly afterwards, and executed "nfsstat -l"

Using the following line in /etc/exports on the server