| View previous topic :: View next topic |
| Author |
Message |
slaterson
Guru
Joined: 26 Feb 2003
Posts: 313
|
 Posted: Sun Aug 15, 2021 12:54 am Post subject: auth on server is failing |
 |
|
i've had a server running for nearly 10 years, hosting my ldap db (among other things). today, i did a typical update that i do almost daily (this time it was about 4 days worth of portage updates) and i can no longer login remotely, nor locally. i noticed nss_ldap was being updated, however lost the other updates.
good news: i have a remote shell open via ssh as root, so i can make updates to config, etc.
bad news: i am not able to login remotely or locally, even as root, to the server. prior to the update today i was able to login locally as anyone, and remotely as anyone except root (su after remote login).
i have reviewed my openldap config and nsswitch.conf. tried a couple tweaks, it all looks sane and matches client config. i'm not sure what else to do other than reboot the server, which may result in me being completely locked out of the server.
i need help. this server has several terabytes of direct attached data. it's a home server - photos, media, etc... |
|
| Back to top |
|
 |
CooSee
Veteran
Joined: 20 Nov 2004
Posts: 1438
Location: Earth
|
| Posted: Sun Aug 15, 2021 2:09 am Post subject: |
|
|
did you also emerged ' sys-apps/shadow ' to 4.9-r1 ?
i got a problem with this version on fresh installed gentoo system couple days ago.
downgrade sys-apps/shadow to 4.8.1-r4.
good luck
_________________
" Die Realität ist eine Illusion, die durch Mangel an ehrlicher Kommunikation entsteht "
---
" Der Mensch ist von Natur aus neugierig, was am Ende übrig bleibt ist die Gier " |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21624
|
| Posted: Sun Aug 15, 2021 3:57 am Post subject: |
|
|
| When authentication fails, what messages are written to the relevant logs? |
|
| Back to top |
|
|
slaterson
Guru
Joined: 26 Feb 2003
Posts: 313
|
| Posted: Sun Aug 15, 2021 1:10 pm Post subject: |
|
|
| CooSee wrote: | did you also emerged ' sys-apps/shadow ' to 4.9-r1 ?
i got a problem with this version on fresh installed gentoo system couple days ago.
downgrade sys-apps/shadow to 4.8.1-r4.
good luck |
sys-apps/shadow was never updated, stayed at 4.8.1-r4. i rebuilt it anyway, and issue persists. |
|
| Back to top |
|
|
slaterson
Guru
Joined: 26 Feb 2003
Posts: 313
|
| Posted: Sun Aug 15, 2021 1:16 pm Post subject: |
|
|
| Hu wrote: | | When authentication fails, what messages are written to the relevant logs? |
in /var/log/auth.log, when trying to su - <user>, i get | Code: | | No passwd entry for user 'chris' |
and... when i tried to login locally as root just now (after rebuilding all packages last night) i am able to successfully. on the server, i am not able to login as any user in my ldap directory, though. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21624
|
| Posted: Sun Aug 15, 2021 4:00 pm Post subject: |
|
|
Does chris exist locally on the failed system or is that user only available through ldap? If chris is through LDAP, what messages are written when you try a console login using a non-LDAP user, preferably root? My inclination is to first try to repair your ability to get a new root shell, then look at getting LDAP in order, once you have the luxury of starting new root shells at will.
If this is not a systemd-based system, you could try restarting sshd. (An old news item leads me to believe that, on at least some systemd-based systems, restarting sshd may kill your connection, and since you cannot get a new one, we want to avoid any risk of killing the existing one.) |
|
| Back to top |
|
|
slaterson
Guru
Joined: 26 Feb 2003
Posts: 313
|
| Posted: Sun Aug 15, 2021 6:19 pm Post subject: |
|
|
| Hu wrote: | | Does chris exist locally on the failed system or is that user only available through ldap? If chris is through LDAP, what messages are written when you try a console login using a non-LDAP user, preferably root? My inclination is to first try to repair your ability to get a new root shell, then look at getting LDAP in order, once you have the luxury of starting new root shells at will. |
chris does not exist locally on the server, only in the ldap directory. i can login locally as root, and i can change sshd_config to allow remote root login (prefer to not to, of course).
auth.log messages when logging as root (successful):
| Code: | Aug 15 11:00:46 bonobo login[28295]: pam_unix(login:session): session opened for user root(uid=0) by LOGIN(uid=0)
Aug 15 11:00:46 bonobo login[4375]: ROOT LOGIN on '/dev/tty1' |
| Quote: | | If this is not a systemd-based system, you could try restarting sshd. (An old news item leads me to believe that, on at least some systemd-based systems, restarting sshd may kill your connection, and since you cannot get a new one, we want to avoid any risk of killing the existing one.) |
(server is openrc, no systemd) i tried restarting sshd at the server (locally logged in). my open remote didn't die, and otherwise no other changes.
if i'm not able to login to the server as an ldap only user, imo definitely points to an ldap issue. the odd thing, i logged in to the server remotely yesterday (as chris), su to root, then updated the system. after updating, i tried to remote login to the server in a new session (do to something else) and it failed. has to be something in the update.
i have tried re-emerging nss_ldap, pam_ldap, and shadow. no luck yet. i have also reviewed nsswitch.conf, ldap.conf, slapd.conf, and pam.d/system-auth configuration files. nothing seems wrong compared to working client configurations.
fwiw, here is the full log entry when trying to login as an ldap only user:
| Code: | Aug 15 11:14:16 bonobo login[4896]: pam_unix(login:auth): check pass; user unknown
Aug 15 11:14:16 bonobo login[4896]: pam_unix(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty1 ruser= rhost=
Aug 15 11:14:16 bonobo login[4896]: pam_faillock(login:auth): User unknown
Aug 15 11:14:16 bonobo last message buffered 1 times
Aug 15 11:14:19 bonobo login[4896]: FAILED LOGIN (1) on '/dev/tty1' FOR 'UNKNOWN', Authentication failure |
edit:
does NOT include any ldap users. on clients, it does.
returns everything from the ldap server, just like it should. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21624
|
| Posted: Sun Aug 15, 2021 7:10 pm Post subject: |
|
|
I agree, this looks like an LDAP issue. In your opening post, you said you could not log in remotely or locally, even as root. From that, I focused on fixing that problem so that you would have the ability to administer the system even if your leftover ssh session was terminated. Logging in locally as root may be less convenient, and I can understand why you want to avoid allowing remote root login, but at least having those as options means that fixing this after that leftover session terminates will not require a liveCD environment.
Everything you've shown leads me to believe that the system no longer lists LDAP users, and your problems flow from there. Unfortunately, I don't know how to turn that observation into further debugging advice. You've already checked all the things I would have suggested. |
|
| Back to top |
|
|
slaterson
Guru
Joined: 26 Feb 2003
Posts: 313
|
| Posted: Sun Aug 15, 2021 7:37 pm Post subject: |
|
|
| Hu wrote: | I agree, this looks like an LDAP issue. In your opening post, you said you could not log in remotely or locally, even as root. From that, I focused on fixing that problem so that you would have the ability to administer the system even if your leftover ssh session was terminated. Logging in locally as root may be less convenient, and I can understand why you want to avoid allowing remote root login, but at least having those as options means that fixing this after that leftover session terminates will not require a liveCD environment.
Everything you've shown leads me to believe that the system no longer lists LDAP users, and your problems flow from there. Unfortunately, I don't know how to turn that observation into further debugging advice. You've already checked all the things I would have suggested. |
thanks for the thoughts thus far, much appreciated. and to be sure it's clear, immediately following the update yesterday i was not able to login locally even as root. now that i can, i feel a little safer in rebooting. still really odd why ldap is not working. i may reboot later today, or later this week, depending on how much time i on my hands to go through emergency recovery.
this server houses home directories for four clients on the home network, if auth to the clients starts failing i'm in trouble and will need to make local users/groups that match the server. |
|
| Back to top |
|
|
alamahant
Advocate
Joined: 23 Mar 2019
Posts: 3879
|
| Posted: Sun Aug 15, 2021 7:48 pm Post subject: |
|
|
Are you using
nss-pam-ldapd
or
sssd
?
sssd is infinitely superior.
It is pam issue most probably or a cert issue
Plz post the output of
| Code: |
cat /etc/pam.d/system-auth
cat /etc/nsswitch.conf
|
from the client.
Also in the client
/etc/openldap/ldap.conf
append
| Code: |
TLS_CACERT /etc/ssl/certs/ca-certificates.crt
TLS_REQCERT allow
|
What is the output from the client(and from the server)
| Code: |
ldapsearch -x -D cn=Manager,dc=my,dc=domain -b dc=my,dc=domain -H ldap://<fqdn-or-IP-of-ldap-server>/ -W
ldapsearch -x -D cn=Manager,dc=my,dc=domain -b dc=my,dc=domain -H ldaps://<fqdn-or-IP-of-ldap-server>/ -W
|
Also if firewall plz open 389 and 636 ports.
if selinux plz set permissive.
_________________
 |
|
| Back to top |
|
|
slaterson
Guru
Joined: 26 Feb 2003
Posts: 313
|
| Posted: Sun Aug 15, 2021 10:52 pm Post subject: |
|
|
| i just masked sys-auth/nss_ldap-265-r9, re-emerged sys-auth/nss_ldap-265-r5 (previous version) and all is well again. at least on my system, -r9 breaks the ldap config, so i'll be leaving it masked. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
