| View previous topic :: View next topic |
| Author |
Message |
KShots
Guru
Joined: 09 Oct 2003
Posts: 597
Location: Florida
|
 Posted: Mon Apr 14, 2025 5:43 pm Post subject: Kerberized NFS4 mystery |
 |
|
I used to run kerberized NFS4 on my network until about a year after the switch from rpc.svcgssd to gssproxy (once configured, this seemed to work reasonably well until that point). At some point, it mysteriously stopped working and until now I've simply disabled kerberos on NFS. I'd like to address that now...
Current configuration:
Server Side
| Code: | gorgon ~ # klist -ke /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
15 host/gorgon.mydomain.com@MYDOMAIN.COM (aes256-cts-hmac-sha1-96) |
| Code: | gorgon ~ # klist -ke /etc/krb5.nfs.keytab
Keytab name: FILE:/etc/krb5.nfs.keytab
KVNO Principal
---- --------------------------------------------------------------------------
11 nfs/gorgon.mydomain.com@MYDOMAIN.COM (aes256-cts-hmac-sha1-96) |
| Code: | gorgon ~ # cat /etc/krb5.conf
[libdefaults]
dns_lookup_realm = true
dns_lookup_kdc = true
ticket_lifetime = 1d
renew_lifetime = 1d
forwardable = true
clockskew = 120
default_tkt_enctypes = aes256-cts-hmac-sha1-96
default_tgs_enctypes = aes256-cts-hmac-sha1-96
permitted_enctypes = aes256-cts-hmac-sha1-96
max_life = 24h
spake_preauth_groups = edwards25519
[realms]
MYDOMAIN.COM = {
admin_server = kerberosadmin.mydomain.com
kdc = kerberos1.mydomain.com
default_domain = mydomain.com
database_module = openldap_ldapconf
disable_encrypted_timestamp = true
}
[dbdefaults]
database_module = openldap_ldapconf
[dbmodules]
openldap_ldapconf = {
db_library = kldap
ldap_kerberos_container_dn = "cn=Kerberos,dc=mydomain,dc=com,o=My Org"
ldap_kdc_dn = "cn=kdc,cn=Kerberos,dc=mydomain,dc=com,o=My Org"
ldap_kadmind_dn = "cn=kadmind,cn=Kerberos,dc=mydomain,dc=com,o=My Org"
ldap_service_password_file = "/var/lib/krb5kdc/kdc-ldap-pwd"
ldap_server = ldap://ldap1.mydomain.com/
ldap_conns_per_server = 2
}
[domain_realm]
.mydomain.com = MYDOMAIN.COM
[logging]
kdc = SYSLOG
admin_server = SYSLOG
default = SYSLOG
[appdefaults]
kinit = {
renewable = true
forwardable = true
}
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
} |
This last configuration (krb5.conf) is shared with all the clients and is identical on the clients.
That said, I run the following to try to get some logging during mount attempts: | Code: | | gorgon ~ # rpc.mountd --debug all --log-auth --foreground |
| Code: | | gorgon /var/log/remote/gorgon # rpc.idmapd -vvvf |
| Code: | | gorgon /var/log/remote/gorgon # gssproxy -i --debug-level=2 |
On this last, I have the following environment variables defined (to match the systemd service file with the alternative keytab): | Code: | gorgon ~ # echo $KRB5CACHEDIR ; echo $KRB5_KTNAME ; echo $GSS_USE_PROXY
/var/lib/gssproxy/rcache
/etc/krb5.nfs.keytab
1 |
| Code: | gorgon ~ # cat /etc/exports.d/home.exports
/home 10.4.12.0/24(rw,crossmnt,subtree_check)
/home 10.4.13.0/24(rw,crossmnt,subtree_check,sec=krb5p) |
The latter network (10.4.13.0/24) is what I'm testing with at this time.
Client Side
| Code: | rich@succubus ~ $ grep gorgon /etc/fstab
gorgon.mydomain.com:/home /mnt/nethome nfs4 noauto,retrans=1000,sec=krb5p 0 0 |
| Code: | rich@succubus ~ $ sudo klist -ke /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
2 host/succubus.mydomain.com@MYDOMAIN.COM (aes256-cts-hmac-sha1-96)
2 nfs/succubus.mydomain.com@MYDOMAIN.COM (aes256-cts-hmac-sha1-96) |
I think I've read somewhere that clients don't really need a nfs/ entry in their keytab, but it shouldn't hurt that there is one defined.
In addition, the machine has kerberos credentials cached: | Code: | rich@succubus ~ $ ls -l /tmp/krb5ccmachine_MYDOMAIN.COM
-rw------- 1 root root 1523 Apr 14 11:21 /tmp/krb5ccmachine_MYDOMAIN.COM |
With this set, and the three services running in the foreground as above, I attempt to mount this share: | Code: | rich@succubus ~ $ sudo mount /mnt/nethome
mount.nfs4: Connection refused for gorgon.mydomain.com:/home on /mnt/nethome |
... and despite all the logging being requested in the above services, nothing shows up in any of the consoles (beyond the initialization messages - nothing showing any connection attempts). What am I missing here?
EDIT:
Interestingly, when not running in the foreground and handed back to systemd, I get the following in my rpc.mountd logs (when I add the below as a dropin to the service): | Code: | # /etc/systemd/system/nfs-mountd.service.d/mountd.conf
[Service]
ExecStart=
ExecStart=/usr/sbin/rpc.mountd --debug all --log-auth
|
when attempting to mount the share: | Code: | Apr 14 14:01:17 gorgon rpc.mountd[2331108]: auth_unix_ip: inbuf 'nfsd 10.4.13.2'
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: v4root_create: path '/' flags 0x12403
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: v4root_create: path '/' flags 0x12403
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: successful authentication for IP 10.4.13.2 as 10.4.13.0/24
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: auth_unix_ip: client 0x55ea7271ec60 '10.4.13.0/24'
fApr 14 14:01:17 gorgon rpc.mountd[2331108]: v4.0 client attached: 0x89c662a367fd4d2f from "10.4.13.2:963"
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: nfsd_fh: inbuf '10.4.13.0/24 1 \x00000000'
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: nfsd_fh: found 0x55ea72721670 path /
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: nfsd_export: inbuf '10.4.13.0/24 /'
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: granted access to / for 10.4.13.0/24
Apr 14 14:01:17 gorgon rpc.mountd[2331108]: nfsd_export: found 0x55ea72721660 path / |
... and the client shows the following when it fails: | Code: | rich@succubus ~ $ sudo mount /mnt/nethome
mount.nfs4: access denied by server while mounting gorgon.mydomain.com:/home |
I also added the below drop-in for gssproxy: | Code: | gorgon /var/log/remote/gorgon # cat /etc/systemd/system/gssproxy.service.d/gorgon.conf
[Service]
Environment=KRB5_KTNAME=/etc/krb5.nfs.keytab
Environment=GSS_USE_PROXY=1
ExecStart=
ExecStart=/usr/sbin/gssproxy -D --debug-level=2 --syslog-status |
... but as far as I can tell, it does not report to syslog (or anywhere else for that matter - journalctl -f shows no activity either on connection attempts) 
EDIT: I found the following in my journal after restarting the nfs-server service: | Code: | | Apr 14 14:29:30 gorgon systemd[1]: RPC security service for NFS server was skipped because no trigger condition checks were met. |
That looks likely to be the problem...
EDIT2: False alarm, that's rpc.svcgssd, which should not be started because gssproxy is running and performing that function.
EDIT3: DNS (forward and reverse) is also important, so below is what I have setup (I had to adjust the client as its DNS changes when running through my VPN on 10.4.13.0/24): | Code: | rich@succubus ~ $ dig rich.mydomain.com
; <<>> DiG 9.18.29 <<>> rich.mydomain.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40636
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1432
;; QUESTION SECTION:
;rich.mydomain.com. IN A
;; ANSWER SECTION:
rich.mydomain.com. 3600 IN A 10.4.13.2
;; Query time: 80 msec
;; SERVER: 10.4.12.10#53(10.4.12.10) (UDP)
;; WHEN: Mon Apr 14 15:12:25 EDT 2025
;; MSG SIZE rcvd: 64
rich@succubus ~ $ dig -x 10.4.13.2
; <<>> DiG 9.18.29 <<>> -x 10.4.13.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41730
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1432
;; QUESTION SECTION:
;2.13.4.10.in-addr.arpa. IN PTR
;; ANSWER SECTION:
2.13.4.10.in-addr.arpa. 3600 IN PTR rich.mydomain.com.
;; Query time: 103 msec
;; SERVER: 10.4.12.10#53(10.4.12.10) (UDP)
;; WHEN: Mon Apr 14 15:12:32 EDT 2025
;; MSG SIZE rcvd: 84
rich@succubus ~ $ sudo klist -k /etc/krb5.keytab
Keytab name: FILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
2 host/succubus.mydomain.com@MYDOMAIN.COM
2 nfs/succubus.mydomain.com@MYDOMAIN.COM
3 host/rich.mydomain.com@MYDOMAIN.COM
2 nfs/rich.mydomain.com@MYDOMAIN.COM
rich@succubus ~ $ dig gorgon.mydomain.com
; <<>> DiG 9.18.29 <<>> gorgon.mydomain.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59373
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1432
;; QUESTION SECTION:
;gorgon.mydomain.com. IN A
;; ANSWER SECTION:
gorgon.mydomain.com. 43200 IN A 10.4.12.19
;; Query time: 83 msec
;; SERVER: 10.4.12.10#53(10.4.12.10) (UDP)
;; WHEN: Mon Apr 14 15:13:30 EDT 2025
;; MSG SIZE rcvd: 66
rich@succubus ~ $ dig -x 10.4.12.19
; <<>> DiG 9.18.29 <<>> -x 10.4.12.19
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49951
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1432
;; QUESTION SECTION:
;19.12.4.10.in-addr.arpa. IN PTR
;; ANSWER SECTION:
19.12.4.10.in-addr.arpa. 43200 IN PTR gorgon.mydomain.com.
;; Query time: 73 msec
;; SERVER: 10.4.12.10#53(10.4.12.10) (UDP)
;; WHEN: Mon Apr 14 15:13:33 EDT 2025
;; MSG SIZE rcvd: 87 |
_________________
Life without passion is death in disguise |
|
| Back to top |
|
 |
KShots
Guru
Joined: 09 Oct 2003
Posts: 597
Location: Florida
|
| Posted: Mon Apr 14, 2025 9:26 pm Post subject: |
|
|
Apparently, none of this was functional until the server was rebooted. Once rebooted, it works fine. Any ideas on what I could have done to save a reboot?
_________________
Life without passion is death in disguise |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
