| View previous topic :: View next topic |
| Author |
Message |
Gatak
Apprentice
Joined: 04 Jan 2004
Posts: 174
|
 Posted: Wed Sep 18, 2019 6:37 pm Post subject: pNFS - parallel NFS setup howto |
 |
|
Hi!
Trying to set up pNFS with the file or flexfile layouts (what's the difference?). However, even though I export with pnfs, mountstats show pnfs=notconfigured (see below).
I am really struggling to find any information on how to set up a pNFS file server. Most information is really old and refers to obsolete spnfsd packages and other things. Can anyone here help?
kernel-5.3.0 on both client and server. Same NFS config too.
net-fs/nfs-utils-2.4.1-r1 on both client and server
SERVER setup:
Kernel config:
| Code: |
┌───────────────────────────────────────────────────────────────────────────────────┐
│ │ --- Network File Systems │ │
│ │ <*> NFS client support │ │
│ │ < > NFS client support for NFS version 2 │ │
│ │ < > NFS client support for NFS version 3 │ │
│ │ <*> NFS client support for NFS version 4 │ │
│ │ [*] Provide swap over NFS support │ │
│ │ [*] NFS client support for NFSv4.1 │ │
│ │ [*] NFS client support for NFSv4.2 │ │
│ │ (kernel.org) NFSv4.1 Implementation ID Domain │ │
│ │ [*] NFSv4.1 client support for migration │ │
│ │ [ ] Use the legacy NFS DNS resolver │ │
│ │ <*> NFS server support │ │
│ │ -*- NFS server support for NFS version 3 │ │
│ │ [*] NFS server support for the NFSv3 ACL protocol extension │ │
│ │ [*] NFS server support for NFS version 4 │ │
│ │ [*] NFSv4.1 server support for pNFS block layouts │ │
│ │ [*] NFSv4.1 server support for pNFS SCSI layouts │ │
│ │ [*] NFSv4.1 server support for pNFS Flex File layouts │ │
│ │ <M> Secure RPC: Kerberos V mechanism │ │
│ │ [ ] Secure RPC: Disable insecure Kerberos encryption types │ │
│ │ [*] RPC: Enable dprintk debugging │ │
│ │ < > Ceph distributed file system │ │
│ │ <M> SMB3 and CIFS support (advanced network filesystem) │ │
│ │ [ ] Extended statistics │ │
│ │ [*] Support legacy servers which use less secure dialects │ │
│ │ [ ] Support legacy servers which use weaker LANMAN security │ │
│ │ [*] Kerberos/SPNEGO advanced session setup │ │
│ │ [*] CIFS extended attributes │ │
│ │ [*] CIFS POSIX Extensions │ │
│ │ [ ] Enable CIFS debugging routines │ │
│ │ [*] DFS feature support │ │
│ └────────────────────────────.(+)───────────────────────────────────────────────┘ │
├───────────────────────────────────────────────────────────────────────────────────┤
│ <Select> < Exit > < Help > < Save > < Load > │
└───────────────────────────────────────────────────────────────────────────────────┘
|
/etc/exports:
| Code: | /mnt/storageTemp/portage/distfiles 192.168.0.50(rw,no_subtree_check,no_root_squash,async,pnfs)
/usr/portage/ 192.168.0.50(rw,no_subtree_check,no_root_squash,async,pnfs)
[/quote] |
dmesg output:
| Code: | # dmesg |grep -i nfs
[ 1.085275] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 1.953873] NFS: Registering the id_resolver key type
[ 1.953912] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 1.953922] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[ 75.592931] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[ 75.593034] NFSD: Using legacy client tracking operations.
[ 75.593039] NFSD: starting 90-second grace period (net f0000098)
|
Client config:
/etc/fstab
| Code: | 192.168.0.1:/usr/portage /usr/portage/ nfs nconnect=2,soft,noatime,rw,tcp,vers=4.2,ac,async,nofail 0 0
192.168.0.1:/mnt/storageTemp/portage/distfiles /usr/portage/distfiles nfs nconnect=2,soft,noatime,rw,tcp,vers=4.2,ac,async,x-systemd.requires=/usr/portage,nofail 0 0 |
mountstats /usr/portage
| Code: | # mountstats /usr/portage/
Stats for 192.168.0.1:/usr/portage mounted on /usr/portage:
NFS mount options: rw,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,soft,proto=tcp,nconnect=2,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.50,local_lock=none
NFS server capabilities: caps=0x7fbffff,wtmult=512,dtsize=32768,bsize=0,namlen=255
NFSv4 capability flags: bm0=0xfdffbfff,bm1=0x40f9be3e,bm2=0x28803,acl=0x3,sessions,pnfs=notconfigured,lease_time=90,lease_expired=0
NFS security flavor: 1 pseudoflavor: 0
NFS byte counts:
applications read 19224685 bytes via read(2)
applications wrote 0 bytes via write(2)
applications read 0 bytes via O_DIRECT read(2)
applications wrote 0 bytes via O_DIRECT write(2)
client read 7128010 bytes via NFS READ
client wrote 0 bytes via NFS WRITE
RPC statistics:
22279 RPC requests sent, 22279 RPC replies received (0 XIDs not found)
average backlog queue length: 0
GETATTR:
22671 ops (101%)
avg bytes sent per op: 227 avg bytes received per op: 240
backlog wait: 0.002470 RTT: 0.513122 total execute time: 0.534912 (milliseconds)
CLOSE:
4530 ops (20%)
avg bytes sent per op: 235 avg bytes received per op: 112
backlog wait: 0.019868 RTT: 0.539735 total execute time: 0.577704 (milliseconds)
DELEGRETURN:
4530 ops (20%)
avg bytes sent per op: 247 avg bytes received per op: 160
backlog wait: 62.107064 RTT: 2.022737 total execute time: 64.153863 (milliseconds)
OPEN_NOATTR:
3513 ops (15%)
avg bytes sent per op: 295 avg bytes received per op: 352
backlog wait: 0.021349 RTT: 0.596356 total execute time: 0.637062 (milliseconds)
LOOKUP:
3163 ops (14%) 258 errors (8%)
avg bytes sent per op: 255 avg bytes received per op: 290
backlog wait: 0.002529 RTT: 0.734745 total execute time: 0.759406 (milliseconds)
ACCESS:
2342 ops (10%)
avg bytes sent per op: 235 avg bytes received per op: 167
backlog wait: 0.002989 RTT: 0.584543 total execute time: 0.608881 (milliseconds)
READ:
2303 ops (10%)
avg bytes sent per op: 243 avg bytes received per op: 3200
backlog wait: 0.020408 RTT: 0.869301 total execute time: 0.911420 (milliseconds)
OPEN:
1056 ops (4%) 39 errors (3%)
avg bytes sent per op: 321 avg bytes received per op: 400
backlog wait: 0.026515 RTT: 1.165720 total execute time: 1.214962 (milliseconds)
READDIR:
370 ops (1%)
avg bytes sent per op: 247 avg bytes received per op: 15366
backlog wait: 0.002703 RTT: 17.191892 total execute time: 17.235135 (milliseconds)
SEQUENCE:
37 ops (0%)
avg bytes sent per op: 116 avg bytes received per op: 80
backlog wait: 0.027027 RTT: 0.891892 total execute time: 0.972973 (milliseconds)
SEEK:
3 ops (0%) 1 errors (33%)
avg bytes sent per op: 244 avg bytes received per op: 104
backlog wait: 0.000000 RTT: 0.333333 total execute time: 0.333333 (milliseconds)
SERVER_CAPS:
2 ops (0%)
avg bytes sent per op: 172 avg bytes received per op: 164
backlog wait: 0.000000 RTT: 0.000000 total execute time: 0.000000 (milliseconds)
EXCHANGE_ID:
2 ops (0%)
avg bytes sent per op: 232 avg bytes received per op: 92
backlog wait: 0.000000 RTT: 0.500000 total execute time: 0.500000 (milliseconds)
NULL:
1 ops (0%)
avg bytes sent per op: 44 avg bytes received per op: 24
backlog wait: 2.000000 RTT: 0.000000 total execute time: 3.000000 (milliseconds)
FSINFO:
1 ops (0%)
avg bytes sent per op: 172 avg bytes received per op: 168
backlog wait: 0.000000 RTT: 0.000000 total execute time: 0.000000 (milliseconds)
PATHCONF:
1 ops (0%)
avg bytes sent per op: 164 avg bytes received per op: 116
backlog wait: 0.000000 RTT: 0.000000 total execute time: 0.000000 (milliseconds)
CREATE_SESSION:
1 ops (0%)
avg bytes sent per op: 192 avg bytes received per op: 124
backlog wait: 0.000000 RTT: 0.000000 total execute time: 0.000000 (milliseconds)
RECLAIM_COMPLETE:
1 ops (0%)
avg bytes sent per op: 124 avg bytes received per op: 88
backlog wait: 0.000000 RTT: 6.000000 total execute time: 6.000000 (milliseconds)
|
|
|
| Back to top |
|
 |
mike155
Advocate
Joined: 17 Sep 2010
Posts: 4438
Location: Frankfurt, Germany
|
| Posted: Wed Sep 25, 2019 12:05 am Post subject: |
|
|
I don't know anything about pnfs and I've never tried it.
But I'm surprised by your '/etc/exports' file:
| Code: | /mnt/storageTemp/portage/distfiles 192.168.0.50(rw,no_subtree_check,no_root_squash,async,pnfs)
/usr/portage/ 192.168.0.50(rw,no_subtree_check,no_root_squash,async,pnfs) |
This looks like a NFSv3 export file. As far as I can see, you want to use NFSv4. Under NFSv4, you are supposed to create an export directory (say '/exports') and bind mount all directories you want to export into that directory. Hence entries in your exports file should all start with that export directory. |
|
| Back to top |
|
|
Gatak
Apprentice
Joined: 04 Jan 2004
Posts: 174
|
| Posted: Thu Sep 26, 2019 7:13 pm Post subject: |
|
|
| I've seen the use of /exports folder structure but nothing that says it must be that way. In any case I connect as vers=4.2 (look at the mountstats output) so it seems to be working there. |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Fri Sep 27, 2019 4:46 am Post subject: |
|
|
| Requiring a virtual root for all NFS4 exports is cargo-culted advice that hasn't been true for over a decade. It'll work fine without it, unless you're trying to export filesystems with no UUID (squashfs/tmpfs) |
|
| Back to top |
|
|
Gatak
Apprentice
Joined: 04 Jan 2004
Posts: 174
|
| Posted: Fri Sep 27, 2019 12:20 pm Post subject: |
|
|
Parallel NFS is the real thing though. I have seen several docs in various about pNFS clients, but so far nothing for how to set up the server correctly  |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
