Suddenly can't use SSH on a headless machine!

Message

KShots · Post by **KShots** » Wed Apr 28, 2004 1:12 pm

Big problems on little server

. I've had this running fine almost without hitches since oh, mid January. I think I have traced the cause of the problem to one simple, absolutely lethal typo.

I meant to type emerge -uDp --verbose world. I actually typed emerge -uDP --verbose world. What's the difference? Well, for one, the 2nd one doesn't pretend anything. For another, it removes any package it seems to think is unnecessary. I saw it was doing something funky, but decided that whatever it was, it would be better to be in a completed state than somewhere in-between by cancelling.

Well, the bottom line is, everything seems to have survived (that I'm aware of, haven't restarted services yet) except sshd (I restarted that one). I try and log in, it asks for a username as normal. Then it asks for a password and always says it's invalid. Checking /var/log/messages, I get a bunch of errors from sshd with PAM modules. What have I done?! This machine is supposed to be headless with administration done via ssh! Does anyone have any suggestions on how to fix this? I really don't want to have to start over...

scout · Post by **scout** » Wed Apr 28, 2004 1:24 pm

first, make a emerge -uD world, then try revdep-rebuild -p and revdep-rebuild on this machine if you can still log on it. I advise not to restart the other services until this is completed ...

Also, make a etc-update ... we never know

KShots · Post by **KShots** » Wed Apr 28, 2004 7:38 pm

Well, I finished emerge -uD world last night and immediately after started revdep-rebuild. I'd think it would be finished by now, but I still can't log in (I'm not there at the moment, so I can't go local). I will try etc-update as soon as I get home. Any other ideas to try when I get back? And thanks for the reply!

KShots · Post by **KShots** » Wed Apr 28, 2004 10:27 pm

Well, I tried that... no dice. revdep-rebuild says I'm missing a "libwrap.so" file. I tried sym-linking it to a file I do have called "libwrap.a", but again no dice. Also, the sshd is reporting that port 22 is already being used by something else... which makes no sense, I see nothing under "ps ax" that could be doing that! Also, just for kicks, I recompiled the kernel (in case that produced the lib)... again, no dice (but it compiled). The only thing I can find that installs something similar to "libwrap.so" is "tcp-wrappers", which installs the .a version. Everything's depending on the .so version, any idea where I can get this?!

Even then, I don't think that solved the PAM problems I'm getting

clar77 · Post by **clar77** » Wed Apr 28, 2004 11:17 pm

KShots wrote: Also, the sshd is reporting that port 22 is already being used by something else... which makes no sense, I see nothing under "ps ax" that could be doing that!

try a:

Code: Select all

 netstat -a | grep 'LISTEN'

to see if / what is blocking 22.

KShots · Post by **KShots** » Thu Apr 29, 2004 12:28 pm

Code: Select all

rich@gorgon rich $ netstat -a | grep LISTEN
tcp        0      0 *:imaps                 *:*                     LISTEN
tcp        0      0 *:32770                 *:*                     LISTEN
tcp        0      0 *:5800                  *:*                     LISTEN
tcp        0      0 localhost:2600          *:*                     LISTEN
tcp        0      0 *:netbios-ssn           *:*                     LISTEN
tcp        0      0 *:5900                  *:*                     LISTEN
tcp        0      0 *:sunrpc                *:*                     LISTEN
tcp        0      0 *:6000                  *:*                     LISTEN
tcp        0      0 *:2601                  *:*                     LISTEN
tcp        0      0 *:2604                  *:*                     LISTEN
tcp        0      0 *:www                   *:*                     LISTEN
tcp        0      0 *:ssh                   *:*                     LISTEN
tcp        0      0 *:https                 *:*                     LISTEN
unix  2      [ ACC ]     STREAM     LISTENING     94341  /tmp/.X11-unix/X0
unix  2      [ ACC ]     STREAM     LISTENING     94388  /tmp/ksocket-rich/kdeinit__0
unix  2      [ ACC ]     STREAM     LISTENING     94390  /tmp/ksocket-rich/kdeinit-:0
unix  2      [ ACC ]     STREAM     LISTENING     94396  /tmp/.ICE-unix/dcop5471-1083237243
unix  2      [ ACC ]     STREAM     LISTENING     94417  /tmp/ksocket-rich/klauncher9N8tNb.slave-socket
unix  2      [ ACC ]     STREAM     LISTENING     2289   /dev/log
unix  2      [ ACC ]     STREAM     LISTENING     94564  /tmp/.ICE-unix/5615
unix  2      [ ACC ]     STREAM     LISTENING     2727   /var/run/cgisock
unix  2      [ ACC ]     STREAM     LISTENING     3521   /tmp/.zebra
unix  2      [ ACC ]     STREAM     LISTENING     3537   /tmp/.ospfd
unix  2      [ ACC ]     STREAM     LISTENING     94493  /tmp/mcop-rich/localhost-15c7-4090e382
rich@gorgon rich $

ssh starts, but it has that odd message. I've started up a VNC server on the machine so I can access it from inside my network (or tunneling through my zebra/ospf tunnel from the internet). I don't like this solution, any other ideas?

clar77 · Post by **clar77** » Thu Apr 29, 2004 1:15 pm

ever use nmap ?

Code: Select all

emerge nmap

then:

Code: Select all

#namp -sS

as root. its a fast portscan that may give you some more insight.

KShots · Post by **KShots** » Mon May 03, 2004 6:15 pm

Well, I just went to a Linux Install-Fest, bringing the server in. After a lot of poking around, it was determined that the problem lay with defective PAM modules. Here's the effects I had:

1. OK, PAM modules are defective, so uninstall PAM... OK, it didn't uninstall the config files, I removed them manually. Now re-install PAM. Now uninstall and re-install SSH. Wow, SSH works!

2. Try to log into the machine locally... fatal PAM error, cannot log in locally (security feature?

)... Decided that's not really what I wanted, and discovered that the only PAM module installed was for SSH. Copied all PAM modules from another machine to this one, and I can log in again... Again, not sure that's quite what I wanted...

3. Now in the current state, I can SSH into the machine and log in locally... but I can't SSH out because of some weird "keyboard interactive:" thing popping up and screwing up the shell. Any ideas?

iq_zero · Post by **iq_zero** » Sun Sep 25, 2005 1:49 pm

3. Now in the current state, I can SSH into the machine and log in locally... but I can't SSH out because of some weird "keyboard interactive:" thing popping up and screwing up the shell. Any ideas?

I had the same problem.. This solved it for me..

Code: Select all

emerge unmerge ssh
emerge openssh