Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
last|head ssh entry and problems , GLSA Support pls
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
dalu
Guru
Guru


Joined: 20 Jan 2003
Posts: 494

PostPosted: Wed Feb 04, 2015 3:43 pm    Post subject: last|head ssh entry and problems , GLSA Support pls Reply with quote

As posted on Google+, perhaps I'll be more lucky here

Right,

last|head
Code:

root     pts/0        2a02:8070:c4c2:2 Wed Feb  4 15:52   still logged in
root     ssh          2a02:8070:c4c2:2 Wed Feb  4 15:52   still logged in
root     pts/0        2a02:8070:c4c2:2 Tue Feb  3 14:33 - 14:51  (00:17)
root     ssh          2a02:8070:c4c2:2 Tue Feb  3 14:33 - 14:51  (00:17)


As you can see there's a pts/0 and a ssh line
it's the same session however.

Occasionally the ssh one died, for whatever reason.
The pts/* session remains active but no responses are sent to the client since ssh is dead.

Using systemd with a custom config, kernel 3.18.5 gentoo-sources

What might be the reason?

For instance, an Archlinux or Debian system for comparison:

Code:

root     pts/0        2a02:8070:c4c2:2 Wed Feb  4 16:06   still logged in
root     pts/0        2a02:8070:c4c2:2 Wed Feb  4 11:19 - 11:19  (00:00)


Last edited by dalu on Thu Feb 12, 2015 7:00 pm; edited 1 time in total
Back to top
View user's profile Send private message
Schnulli
Guru
Guru


Joined: 25 Jun 2010
Posts: 320
Location: Bremen DE

PostPosted: Sat Feb 07, 2015 1:58 pm    Post subject: Reply with quote

Hi
i am wondering also for the same thing since a while and yet no answer why a dead ssh session is kept somehow alive but dead.....
just wondering ;)
A new Bug?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13509

PostPosted: Sat Feb 07, 2015 4:16 pm    Post subject: Reply with quote

Does the server system know the client has died? If the client process exited while unable to communicate with the server, then the server will not detect this until it tries and fails to send traffic to the client. Generate some output on that pty and the session should go away.
Back to top
View user's profile Send private message
Schnulli
Guru
Guru


Joined: 25 Jun 2010
Posts: 320
Location: Bremen DE

PostPosted: Sat Feb 07, 2015 5:41 pm    Post subject: Reply with quote

Hi Hu

dosnt matter or stresses me..... this means to me, an open pipe is kept... so... reason one... maybe a bug in ssh itself, or reason tow, ssh is not correctly configured
i for myself kill this session by myself.
i never had intentions to find out what the reason is because i screen my systems and kill pids if they are dead
Back to top
View user's profile Send private message
dalu
Guru
Guru


Joined: 20 Jan 2003
Posts: 494

PostPosted: Thu Feb 12, 2015 6:02 pm    Post subject: Reply with quote

Ok, it just happened again.

Funny thing is
As soon as I noticed I opened a new terminal on the client and ssh'd into the server.
did
w
Code:

# w
 18:51:35 up  3:09,  2 users,  load average: 0.10, 0.17, 0.11
USER     TTY        LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0     16:16   15.00s  0.06s  0.06s -bash
root     pts/1     18:51    1.00s  0.01s  0.00s w


I hit [Enter] like 30 times (on pts/0) and the session re-appeared or better, the terminal became responsive again.
However I'm not sure if that's because pts/1 is active.

Code:

# last|head
root     pts/1        2a02:8070:c4c2:2 Thu Feb 12 18:51   still logged in
root     ssh          2a02:8070:c4c2:2 Thu Feb 12 18:51   still logged in
root     pts/0        2a02:8070:c4c2:2 Thu Feb 12 16:16   still logged in
root     ssh          2a02:8070:c4c2:2 Thu Feb 12 16:16 - 18:51  (02:35)


Hu,
yes when the client terminal gets closed the server detects this and "logs off" pts/0 (for instance).

It often happens when I emerge something.

edit:
I have now closed pts/1 session
Code:

root     pts/1        2a02:8070:c4c2:2 Thu Feb 12 18:51 - 19:06  (00:14)
root     ssh          2a02:8070:c4c2:2 Thu Feb 12 18:51 - 19:06  (00:14)
root     pts/0        2a02:8070:c4c2:2 Thu Feb 12 16:16   still logged in
root     ssh          2a02:8070:c4c2:2 Thu Feb 12 16:16 - 18:51  (02:35)

but pts/0 session is still active and responsive (on my client and the server)
Back to top
View user's profile Send private message
Schnulli
Guru
Guru


Joined: 25 Jun 2010
Posts: 320
Location: Bremen DE

PostPosted: Thu Feb 12, 2015 6:42 pm    Post subject: Reply with quote

Hi dalu
the same simptoms here.....
even killing the PID dosnt affect and its still alive......
possible a Bug and new Backdoor detected? Seems like....again ^^

Hey GLSA ! have a look at ;)

@dalu
change ur Topic and add "GLSA Support pls"


Regards
Back to top
View user's profile Send private message
dalu
Guru
Guru


Joined: 20 Jan 2003
Posts: 494

PostPosted: Thu Feb 12, 2015 6:59 pm    Post subject: Reply with quote

You think, Schnulli?
Isn't that kind of drastic before doing our own investigation?

Ah well, better safe than sorry, I don't want to bug them if it isn't necessary :)
They probably have more stuff to worry about.
However yeah it is the default way it works right now (sshd).

What I did so far was
check sshd_config, nothing out of the ordinary
change sshd.service sshd@.service to Archlinux ones add sshgenkeys.service, no effect

Next up:
find out which package wtmp btmp is :D
emerge openssh-6.7 (no -r3, aka without the x509 patch and the other glue patch, not sure what it does) and see if that also happens

netstat doesn't show any other connections for sshd but that doesn't have to mean anything.
Back to top
View user's profile Send private message
Schnulli
Guru
Guru


Joined: 25 Jun 2010
Posts: 320
Location: Bremen DE

PostPosted: Thu Feb 12, 2015 7:06 pm    Post subject: Reply with quote

yep
better safe than sorry is also my way of thinking, thats one reason why Gentoo is still so clean and so much masked because dirty ;)
A backstep is mostly this i use, in this case to risky, the old sshd version is buggy ^^
If you ntop/stat something let it run for at last a month to be sure and >> (pipe) the output it into a file..... file size dosnt matter in our linux thinking mind ;)
by the way, why not to deep and longterm screen with Wireshark ?

regards



dalu wrote:
You think, Schnulli?
Isn't that kind of drastic before doing our own investigation?

Ah well, better safe than sorry, I don't want to bug them if it isn't necessary :)
They probably have more stuff to worry about.
However yeah it is the default way it works right now (sshd).

What I did so far was
check sshd_config, nothing out of the ordinary
change sshd.service sshd@.service to Archlinux ones add sshgenkeys.service, no effect

Next up:
find out which package wtmp btmp is :D
emerge openssh-6.7 (no -r3, aka without the x509 patch and the other glue patch, not sure what it does) and see if that also happens

netstat doesn't show any other connections for sshd but that doesn't have to mean anything.
Back to top
View user's profile Send private message
dalu
Guru
Guru


Joined: 20 Jan 2003
Posts: 494

PostPosted: Thu Feb 12, 2015 7:31 pm    Post subject: Reply with quote

Schnulli wrote:

If you ntop/stat something let it run for at last a month to be sure and >> (pipe) the output it into a file..... file size dosnt matter in our linux thinking mind ;)

Well my rootfs is just 384GB, the rest is dedicated to mongodb.

Schnulli wrote:

by the way, why not to deep and longterm screen with Wireshark ?


I need to write my auth server and lib (http), this wireshark logging would require 1-2 days extra, time is ticking, each day costs 4€ for running the 3 servers and they're not generating any income yet and there's still so much left to do, in short I don't really believe it's a security issue but an annoying bug and I need to get on with my plan :) So much to do, so little time.

Maybe I misunderstood you, I don't like the limited scrollback of screen. And it works on other distros, so it should be working here.

I'll try a modified ebuild without the x509 patches. How is the non-r3 buggy?
Code:

diff openssh-6.7_p1.ebuild openssh-6.7_p1-r3.ebuild
3c3
< # $Header: /var/cvsroot/gentoo-x86/net-misc/openssh/openssh-6.7_p1.ebuild,v 1.13 2014/12/31 07:40:01 vapier Exp $
---
> # $Header: /var/cvsroot/gentoo-x86/net-misc/openssh/openssh-6.7_p1-r3.ebuild,v 1.2 2014/12/31 07:29:47 vapier Exp $
14c14
< #X509_VER="8.1" X509_PATCH="${PARCH/6.7/6.6}+x509-${X509_VER}.diff.gz"
---
> X509_VER="8.2" X509_PATCH="${PARCH}+x509-${X509_VER}.diff.gz"
31,32c31,32
< KEYWORDS="alpha amd64 arm arm64 hppa ia64 m68k ~mips ppc ppc64 s390 sh sparc x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd ~arm-linux ~x86-linux"
< IUSE="bindist ${HPN_PATCH:++}hpn kerberos ldap ldns libedit pam +pie sctp selinux skey static X X509"
---
> KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd ~arm-linux ~x86-linux"
> IUSE="bindist ${HPN_PATCH:++}hpn kerberos kernel_linux ldap ldns libedit pam +pie sctp selinux skey static X X509"
110,111c110,111
<       epatch "${FILESDIR}"/${PN}-6.6.1_p1-x509-glue.patch
<       use hpn && epatch "${FILESDIR}"/${PN}-6.6.1_p1-x509-hpn14v5-glue.patch
---
>       epatch "${FILESDIR}"/${P}-x509-glue.patch
>       epatch "${FILESDIR}"/${P}-sctp-x509-glue.patch
191c191
<       --with-pid-dir="${EPREFIX}"/var/run \
---
>       --with-pid-dir="${EPREFIX}"$(usex kernel_linux '' '/var')/run \

Have you tried "downgrading" to the non-r3 ebuild?
Back to top
View user's profile Send private message
Schnulli
Guru
Guru


Joined: 25 Jun 2010
Posts: 320
Location: Bremen DE

PostPosted: Thu Feb 12, 2015 7:38 pm    Post subject: Reply with quote

yes, on other distros it works... thats why i ask myself what the heck is on Gentoo the reason.....

No income yet on ur servers? where they are located? Country? and Line speed? Any Traffic limitations?
Lets talk and open me a VirtualBox Slot & v-host remote management and i ll help paying ur bills at last a few ;)
This is more easy instead of renting again another 4HE Rack slot somewhere

No i havent downgraded yet bec. my SSHs are running behind a firewall, thats the reason why i wonder but dont get scared ;)

dalu wrote:
Schnulli wrote:

If you ntop/stat something let it run for at last a month to be sure and >> (pipe) the output it into a file..... file size dosnt matter in our linux thinking mind ;)

Well my rootfs is just 384GB, the rest is dedicated to mongodb.

Schnulli wrote:

by the way, why not to deep and longterm screen with Wireshark ?


I need to write my auth server and lib (http), this wireshark logging would require 1-2 days extra, time is ticking, each day costs 4€ for running the 3 servers and they're not generating any income yet and there's still so much left to do, in short I don't really believe it's a security issue but an annoying bug and I need to get on with my plan :) So much to do, so little time.

Maybe I misunderstood you, I don't like the limited scrollback of screen. And it works on other distros, so it should be working here.

I'll try a modified ebuild without the x509 patches. How is the non-r3 buggy?
Code:

diff openssh-6.7_p1.ebuild openssh-6.7_p1-r3.ebuild
3c3
< # $Header: /var/cvsroot/gentoo-x86/net-misc/openssh/openssh-6.7_p1.ebuild,v 1.13 2014/12/31 07:40:01 vapier Exp $
---
> # $Header: /var/cvsroot/gentoo-x86/net-misc/openssh/openssh-6.7_p1-r3.ebuild,v 1.2 2014/12/31 07:29:47 vapier Exp $
14c14
< #X509_VER="8.1" X509_PATCH="${PARCH/6.7/6.6}+x509-${X509_VER}.diff.gz"
---
> X509_VER="8.2" X509_PATCH="${PARCH}+x509-${X509_VER}.diff.gz"
31,32c31,32
< KEYWORDS="alpha amd64 arm arm64 hppa ia64 m68k ~mips ppc ppc64 s390 sh sparc x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd ~arm-linux ~x86-linux"
< IUSE="bindist ${HPN_PATCH:++}hpn kerberos ldap ldns libedit pam +pie sctp selinux skey static X X509"
---
> KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~amd64-fbsd ~sparc-fbsd ~x86-fbsd ~arm-linux ~x86-linux"
> IUSE="bindist ${HPN_PATCH:++}hpn kerberos kernel_linux ldap ldns libedit pam +pie sctp selinux skey static X X509"
110,111c110,111
<       epatch "${FILESDIR}"/${PN}-6.6.1_p1-x509-glue.patch
<       use hpn && epatch "${FILESDIR}"/${PN}-6.6.1_p1-x509-hpn14v5-glue.patch
---
>       epatch "${FILESDIR}"/${P}-x509-glue.patch
>       epatch "${FILESDIR}"/${P}-sctp-x509-glue.patch
191c191
<       --with-pid-dir="${EPREFIX}"/var/run \
---
>       --with-pid-dir="${EPREFIX}"$(usex kernel_linux '' '/var')/run \

Have you tried "downgrading" to the non-r3 ebuild?
Back to top
View user's profile Send private message
dalu
Guru
Guru


Joined: 20 Jan 2003
Posts: 494

PostPosted: Thu Feb 12, 2015 8:21 pm    Post subject: Reply with quote

Schnulli wrote:
yes, on other distros it works... thats why i ask myself what the heck is on Gentoo the reason.....

No income yet on ur servers? where they are located? Country? and Line speed? Any Traffic limitations?
Lets talk and open me a VirtualBox Slot & v-host remote management and i ll help paying ur bills at last a few ;)
This is more easy instead of renting again another 4HE Rack slot somewhere

No i havent downgraded yet bec. my SSHs are running behind a firewall, thats the reason why i wonder but dont get scared ;)


Downgrading changed nothing :)

I'm not doing virtualization but I can give you the Google+ Profile of someone who just started like a month ago, with 2 E5s using ganetti (and Funtoo).
The 3 servers are cheap refurbished Hetzner Xeon 1245v2 with 16GB ECC RAM and 2x3TB HGST Disks, really low cost.
I'm writing programs in Go since 1¼ years, switched from PHP and picked MongoDB for storage. The 3 cheap ones are cheaper than 1 big server.
I have
3 nginx on the front,
libreswan (IPsec) on private addrs
services listening on private addrs, for instance domain.tld service listening on s0:10000 s1:10000 s2:10000
where
s0 = 10.0.1.1
s1 = 10.0.1.2
s2 = 10.0.1.3
and in the back I have 3 replica sets which form a shard and the services talk to the mongos (mongodb shard service)
since mongodb is able to work with files (gridfs) I use this for storage
All services run as their own user with their custom, very limited shell that only accepts git pushes and "update" "build" "env" "ls".
And I have a management service listening on each server to create those services/users.
No virtualization, no containers, just systemd settings to control read/write permissions and capabilities and limits.
Since each service is a static binary that runs in its VM with safe types and 1 "GOMAXPROC" I'm not afraid of off by 1 or other attacks and the mongodb driver sanitizes by default (also because of safe Go types), so no fear of "injections".
And it all costs ~120€ / month.
That's 7.8TB single replicated storage, where 2 nodes can come down and content is still serving, but no writes can be made.
Bandwidth on paper is limited to 200mbit/s per node, real data shows it's less.
Each node is limited to 20TB outgoing / month, good enough for me, for starters :)
So that's 12 cores, 3x16GB ECC RAM, 7.8TB replicated storage across nodes and theoretical throughput of 600mbit/s for 120€ / month.
Aka ~5k-15k concurrent connections.
A similar offer from online.net by with just 2x3TB storage costs 155€ incl VAT.
And I don't have to deal with VMs and/or containerization.

So for all my domains I first need to write the base, authentication
then add authorization
then make it openid-connect compatible
and lastly build content sites or "apps"
but all white-hat and legal, no black-hat stuff, nothing illegal.

You know what I'm looking for? Audio ads for mp3 content. If I could monetize audio I could afford to pay people to do coding and travel the world, instead of sitting in front of my PC the whole day long :) And no Youtube doesn't cut it.

I can send you a PM if you want the guy's contact on Google+
Back to top
View user's profile Send private message
Schnulli
Guru
Guru


Joined: 25 Jun 2010
Posts: 320
Location: Bremen DE

PostPosted: Thu Feb 12, 2015 10:18 pm    Post subject: Reply with quote

u have a PN ;)
Back to top
View user's profile Send private message
dalu
Guru
Guru


Joined: 20 Jan 2003
Posts: 494

PostPosted: Mon Feb 16, 2015 12:08 pm    Post subject: Reply with quote

Now back on topic.

Code:

last|head
root     pts/1        2a02:8070:c4c2:2 Mon Feb 16 12:38   still logged in
root     ssh          2a02:8070:c4c2:2 Mon Feb 16 12:38   still logged in
root     pts/0        2a02:8070:c4c2:2 Mon Feb 16 11:59   still logged in
root     ssh          2a02:8070:c4c2:2 Mon Feb 16 11:59 - 12:38  (00:38)


I had "less /etc/pam.d/system-auth" running and my ISP decided that it's time to reboot my router (...),
so I got disconnected. Logged in again and pts/0 was still active with "less" still running.
So I killed less and killed bash associated with pts/0.

I've noticed that there's a difference between Archlinux' and Gentoo's /etc/pam.d/systemd-user
and few others

/etc/pam.d/sshd
points to system-remote-login
system-remote-login points to system-login

Gentoo's system-login
Code:

auth            required        pam_tally2.so onerr=succeed
auth            required        pam_shells.so
auth            required        pam_nologin.so
auth            include         system-auth
account         required        pam_access.so
account         required        pam_nologin.so
account         include         system-auth
account         required        pam_tally2.so onerr=succeed
password        include         system-auth
session         optional        pam_loginuid.so
session         required        pam_env.so
session         optional        pam_lastlog.so silent
session         include         system-auth
session         optional        pam_motd.so motd=/etc/motd
session         optional        pam_mail.so


Archlinux' system-login
Code:

#%PAM-1.0

auth       required   pam_tally.so         onerr=succeed file=/var/log/faillog
auth       required   pam_shells.so
auth       requisite  pam_nologin.so
auth       include    system-auth

account    required   pam_access.so
account    required   pam_nologin.so
account    include    system-auth

password   include    system-auth

session    optional   pam_loginuid.so
session    include    system-auth
session    optional   pam_motd.so          motd=/etc/motd
session    optional   pam_mail.so          dir=/var/spool/mail standard quiet
-session   optional   pam_systemd.so
session    required   pam_env.so


difference
Gentoo has
Code:

auth            required        pam_nologin.so
account         required        pam_tally2.so onerr=succeed

Archlinux has
Code:

auth       requisite  pam_nologin.so
-session   optional   pam_systemd.so


However
system-auth

Gentoo's system-auth
Code:

auth            required        pam_env.so
auth            required        pam_unix.so try_first_pass likeauth nullok
auth            optional        pam_permit.so
account         required        pam_unix.so
account         optional        pam_permit.so
password        required        pam_cracklib.so difok=2 minlen=8 dcredit=2 ocredit=2 retry=3
password        required        pam_unix.so try_first_pass use_authtok nullok sha512 shadow
password        optional        pam_permit.so
session         required        pam_limits.so
session         required        pam_env.so
session         required        pam_unix.so
session         optional        pam_permit.so
-session        optional        pam_systemd.so


Archlinux' system-auth
Code:

#%PAM-1.0

auth      required  pam_unix.so     try_first_pass nullok
auth      optional  pam_permit.so
auth      required  pam_env.so

account   required  pam_unix.so
account   optional  pam_permit.so
account   required  pam_time.so

password  required  pam_unix.so     try_first_pass nullok sha512 shadow
password  optional  pam_permit.so

session   required  pam_limits.so
session   required  pam_unix.so
session   optional  pam_permit.so


Difference ,amongst other things
Gentoo's has
Code:

-session        optional        pam_systemd.so


Now systemd-user

Gentoo's systemd-user
Code:

# This file is part of systemd.
#
# Used by systemd --user instances.

account  include system-auth
session  include system-auth


Archlinux' systemd-user
Code:

# This file is part of systemd.
#
# Used by systemd --user instances.

account  include system-login
session  include system-login


Gentoo wants system-auth
Arch wants system-login

Downloading and installing Fedora to see how they do it.
Also I should probably read pam's manual :)
http://www.linux-pam.org/Linux-PAM-html/Linux-PAM_SAG.html

Fedora 21 uses authconf to generate pam config files (good idea actually).
and its sshd file looks like this
Code:

#%PAM-1.0
auth       required     pam_sepermit.so
auth       substack     password-auth
auth       include      postlogin
# Used with polkit to reauthorize users in remote sessions
-auth      optional     pam_reauthorize.so prepare
account    required     pam_nologin.so
account    include      password-auth
password   include      password-auth
# pam_selinux.so close should be the first session rule
session    required     pam_selinux.so close
session    required     pam_loginuid.so
# pam_selinux.so open should only be followed by sessions to be executed in the user context
session    required     pam_selinux.so open env_params
session    optional     pam_keyinit.so force revoke
session    include      password-auth
session    include      postlogin
# Used with polkit to reauthorize users in remote sessions
-session   optional     pam_reauthorize.so prepare


and the password-auth substack
Code:

#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth        required      pam_env.so
auth        sufficient    pam_unix.so nullok try_first_pass
auth        requisite     pam_succeed_if.so uid >= 1000 quiet_success
auth        required      pam_deny.so

account     required      pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid < 1000 quiet
account     required      pam_permit.so

password    requisite     pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
-session     optional      pam_systemd.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     required      pam_unix.so
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5593

PostPosted: Mon Feb 16, 2015 6:08 pm    Post subject: Reply with quote

Can you justify having PAM installed at all, when you don't understand its security implications?
Back to top
View user's profile Send private message
dalu
Guru
Guru


Joined: 20 Jan 2003
Posts: 494

PostPosted: Tue Mar 03, 2015 10:08 am    Post subject: Reply with quote

Code:

root     pts/0        2a02:8070:c48f:3 Tue Mar  3 09:46   still logged in
root     ssh          2a02:8070:c48f:3 Tue Mar  3 09:46   still logged in
root     pts/2        2a02:8070:c48f:3 Mon Mar  2 20:44 - 20:57  (00:13)
root     ssh          2a02:8070:c48f:3 Mon Mar  2 20:44 - 20:57  (00:12)


Where is pts/1 ?

net-misc/openssh-6.7_p1-r4::gentoo
sys-apps/systemd-219-r1:0/2::gentoo

seriously, what's going on there?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum