Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Login not possible anymore
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
Jimini
Guru
Guru


Joined: 31 Oct 2006
Posts: 594
Location: Germany

PostPosted: Sat Jun 01, 2019 6:32 am    Post subject: Login not possible anymore Reply with quote

Hi folks,

from time to time I have the problem on my local file server, that logging in (locally and via SSH) is not possible anymore. Since rebooting it is not an acceptable solution for me, I'd like to dig into this problem :)
- Login via SSH does not work. I enter the password / key pass phrase and then the connection times out after ~2 minutes with "packet_write_wait: Connection to 10.0.0.2 port 22: Broken pipe". See output of ssh -vvv and my sshd_config attached under this posting.
- Local login does not work. I enter the password, see the time of the last login and nothing happens - no prompt or something like that.
- Remote logging does not work anymore. The system's syslog-ng is connected to my log server, but no data is transmitted.
- The problem occurred for about three times during the last six months. I can not correlate it with changes on the system.
- The system responds to ping and HTTPS requests.
- The system is monitored with Zabbix, which is complaining about network errors since the moment from which on no log data is transmitted anymore.
- According to Zabbix, enough disk space is available.

Of course I can not tell you about the exact software versions I use on this system, since I am unable to look them up right now. I use Gentoo with a 4.19 kernel and the latest stable version of syslog-ng and openssh.

ssh -vvv
Code:
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /home/jimini/.ssh/config
debug1: /home/jimini/.ssh/config line 1: Applying options for 10.0.0.2
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "10.0.0.2" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 10.0.0.2 [10.0.0.2] port 22.
debug1: Connection established.
debug1: identity file /home/jimini/.ssh/privkey type 0
debug1: key_load_public: No such file or directory
debug1: identity file /home/jimini/.ssh/privkey-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9
debug1: match: OpenSSH_7.9 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 10.0.0.2:22 as 'jimini'
debug3: hostkeys_foreach: reading file "/home/jimini/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/jimini/.ssh/known_hosts:25
debug3: load_hostkeys: loaded 1 keys from 10.0.0.2
debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ssh-rsa SHA256:r7IFL7RaEBD79czOWMLRSHqfBeMzYtvDP2SvWZyZ3i8
debug3: hostkeys_foreach: reading file "/home/jimini/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/jimini/.ssh/known_hosts:25
debug3: load_hostkeys: loaded 1 keys from 10.0.0.2
debug1: Host '10.0.0.2' is known and matches the RSA host key.
debug1: Found key in /home/jimini/.ssh/known_hosts:25
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /home/jimini/.ssh/privkey (0x556f0d54b200), explicit, agent
debug2: key: /home/jimini/.ssh/privkey2 (0x556f0d55b460), agent
debug2: key: jimini@Deimos (0x556f0d55b500), agent
debug2: key: jimini@Phobos (0x556f0d55bde0), agent
debug2: key: jimini@Phobos (0x556f0d55bf00), agent
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:cJ2lDrsi2scaqqGcT9gi9iqYp+ImsPmFuRUsaKub6M4 /home/jimini/.ssh/privkey
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: pkalg rsa-sha2-512 blen 279
debug2: input_userauth_pk_ok: fp SHA256:cJ2lDrsi2scaqqGcT9gi9iqYp+ImsPmFuRUsaKub6M4
debug3: sign_and_send_pubkey: RSA SHA256:cJ2lDrsi2scaqqGcT9gi9iqYp+ImsPmFuRUsaKub6M4
debug3: send packet: type 50
debug3: receive packet: type 52
debug1: Authentication succeeded (publickey).
Authenticated to 10.0.0.2 ([10.0.0.2]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: send packet: type 1
packet_write_wait: Connection to 10.0.0.2 port 22: Broken pipe


sshd_config
Code:
Port 12345
ListenAddress 10.0.0.2
Protocol 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key

#UsePrivilegeSeparation yes
#KeyRegenerationInterval 3600
#ServerKeyBits 768
SyslogFacility AUTH
LogLevel INFO
LoginGraceTime 120
PermitRootLogin no
StrictModes yes
PubkeyAuthentication yes
IgnoreRhosts yes
HostbasedAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
PasswordAuthentication no
X11Forwarding no
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive no
MaxAuthTries 2
MaxSessions 2
AllowAgentForwarding NO
AllowTcpForwarding NO

KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

UsePAM no

Subsystem sftp /usr/lib64/misc/sftp-server

ClientAliveInterval 600
ClientAliveCountMax 2
GatewayPorts yes
AllowTcpForwarding no


Any kind of assistance is welcome! :)

Kind regards,
Jimini
_________________
"The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents." (H.P. Lovecraft: The Call of Cthulhu)
Back to top
View user's profile Send private message
Jimini
Guru
Guru


Joined: 31 Oct 2006
Posts: 594
Location: Germany

PostPosted: Sun Jun 02, 2019 6:23 am    Post subject: Reply with quote

I forgot to mention, that this system runs a LXC container with another Gentoo installation on it. This system also offers a SSH login, which does work normally.

King regards,
Jimini
_________________
"The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents." (H.P. Lovecraft: The Call of Cthulhu)
Back to top
View user's profile Send private message
alamahant
Apprentice
Apprentice


Joined: 23 Mar 2019
Posts: 152

PostPosted: Tue Jun 04, 2019 8:43 pm    Post subject: Reply with quote

May be here is the problem?
Code:

TCPKeepAlive no


May be try "yes" instead?

Furthermore
Code:

UsePAM no

It would be better if you used PAM with ssh.

From a centos sshd_config file:
Quote:


# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
# WARNING: 'UsePAM no' is not supported in Red Hat Enterprise Linux and may cause several
# problems.
UsePAM yes
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6356

PostPosted: Wed Jun 05, 2019 3:39 am    Post subject: Reply with quote

Your sshd_config says port 12345 but you're trying to connect to port 22. You're probably not connecting to the sshd you think you are.
Back to top
View user's profile Send private message
Jimini
Guru
Guru


Joined: 31 Oct 2006
Posts: 594
Location: Germany

PostPosted: Wed Jun 05, 2019 5:37 am    Post subject: Reply with quote

Thank you for your replies. I assume, that the SSHd config should not be the problem, since local login does not work either. The connection to the SSHd seems to work, but for me it seems as if the system is unable to provide a shell.

Regarding the port: I have an iptables rule, which forwards tcp 22 to tcp 12345.

Kind regards,
Jimini
_________________
"The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents." (H.P. Lovecraft: The Call of Cthulhu)
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6356

PostPosted: Wed Jun 05, 2019 4:56 pm    Post subject: Reply with quote

If the syslogger can't empty its buffers, then it's possible sshd is blocking when calling syslog(3). Is syslog-ng trying to print logs to its stdout? If that's the case and it has nowhere to write to, it'll eventually back up and the whole system will stop. I don't use syslog-ng, but I've had that happen with metalog.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum