Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[solved] Cannot ssh to other hosts after openssh-7.7_p1-r4
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2  
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
quinn_jones
n00b
n00b


Joined: 14 Dec 2005
Posts: 50
Location: Wilmington, MA, US

PostPosted: Thu Jun 21, 2018 11:27 am    Post subject: Reply with quote

Hi Whissi,

Whissi wrote:
Please try setting IPQoS to a different value, i.e.

Code:
ssh -o IPQoS=lowdelay
or
Code:
ssh -o IPQoS=throughput
and tell us if this will fix your problem.


That does indeed seem to fix the problem. Either one works. Thank you!

Is this considered an actual fix, or just part of the diagnosis?
Back to top
View user's profile Send private message
Whissi
Retired Dev
Retired Dev


Joined: 12 Jan 2011
Posts: 222

PostPosted: Thu Jun 21, 2018 11:57 am    Post subject: Reply with quote

No, I am considering your problem (bunder) "fixed". I.e. your problem were/is caused by a bad agent.

I am especially talking to quinn_jones or anyone else with VMware and who cannot connect anymore since upgraded to Gentoo's OpenSSH 7.7p1 ebuild.


@ quinn_jones: Thanks for confirming. This is clearly a bug in VMware. Thanks to https://bugs.gentoo.org/658488#c3 I have seen captures showing RST packages from VMware when AF21/CS1 DSCP flags are used (this has nothing to do with OpenSSH, you can reproduce with everything.. you just need to set QoS marker to AF21/CS1 for something and VMware will kill that connection via RST package)

So if you have a valid VMware contract please file a bug against upstream. Would be nice if you could share ticket number.

This won't go away so you will need to keep IPQoS setting for now until VMware fixes their product.
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Thu Jun 21, 2018 12:06 pm    Post subject: Reply with quote

quinn_jones wrote:
Is this considered an actual fix, or just part of the diagnosis?

quinn_jones ... a fix, because, if I understand what's happening, it's your router/gateway causing this, ssh sets a QoS (quality of service) packet on TCP, and that flag is probably blocked/interrupted. With 'lowdelay' or 'throughput' that is no longer the case (those are probably ignored). You can test that hypothesis by proxying ssh through netcat ('nc', 'ncat', or one of the other various "network swiss army knife" clients), ie (with net-analyzer/nmap[ncat]):

Code:
% ssh -o ProxyCommand="ncat %h %p" user@host

If that similarly passes through the gateway then it's probably the default QoS ssh sets and your fix is to set IPQoS in ~/.ssh/config, eg:

~/.ssh/config:
Host *
    IPQoS=throughput

I should ask, do you have access to the router/gateway?

best ... khay
Back to top
View user's profile Send private message
bunder
Bodhisattva
Bodhisattva


Joined: 10 Apr 2004
Posts: 5934

PostPosted: Thu Jun 21, 2018 12:13 pm    Post subject: Reply with quote

Whissi wrote:
No, I am considering your problem (bunder) "fixed". I.e. your problem were/is caused by a bad agent.


I don't mean to derail, but what am I supposed to do to fix it then? I'm already using gnupg 2.2.8 (see bug post)
_________________
Neddyseagoon wrote:
The problem with leaving is that you can only do it once and it reduces your influence.

banned from #gentoo since sept 2017
Back to top
View user's profile Send private message
Whissi
Retired Dev
Retired Dev


Joined: 12 Jan 2011
Posts: 222

PostPosted: Thu Jun 21, 2018 12:18 pm    Post subject: Reply with quote

bunder wrote:
I don't mean to derail, but what am I supposed to do to fix it then? I'm already using gnupg 2.2.8 (see bug post)
I don't know your system. You first have to understand what's answering agent calls. I quoted the gnupg bug due to the quote of the problem. This doesn't only affect gnupg's agent. So there's a chance that you are using a different agent which also needs a fix. We would need to identify first :)

Like said, you can test and confirm that it is an agent problem when you just use "ssh" and bypass any agent using arguments like shown.
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
bunder
Bodhisattva
Bodhisattva


Joined: 10 Apr 2004
Posts: 5934

PostPosted: Thu Jun 21, 2018 12:44 pm    Post subject: Reply with quote

it's probably too big to post here, so i threw it up on gists.

https://gist.github.com/bunder2015/ca72ce36c3a9d2a197595d67b6a6e74f

both systems are just regular desktops running mate. i've never installed any ssh agents, except for gnome-keyring which was a dependancy.
_________________
Neddyseagoon wrote:
The problem with leaving is that you can only do it once and it reduces your influence.

banned from #gentoo since sept 2017
Back to top
View user's profile Send private message
Whissi
Retired Dev
Retired Dev


Joined: 12 Jan 2011
Posts: 222

PostPosted: Thu Jun 21, 2018 1:05 pm    Post subject: Reply with quote

We don't have gnome-keyring with https://gitlab.gnome.org/GNOME/gnome-keyring/commit/35a01f8c6eaf3c991aaeb3f66449f41d3f0580bc in repository. I don't know how this program works, if it is hooking someway into ssh even if we call it with "-f" to bypass agent...

Can you make sure gnome-keyring isn't running when testing? Or maybe with some luck, throw the patch into /etc/portage/patches and recompile the package and hope that the patch will apply...?
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
bunder
Bodhisattva
Bodhisattva


Joined: 10 Apr 2004
Posts: 5934

PostPosted: Thu Jun 21, 2018 1:17 pm    Post subject: Reply with quote

sorry i should have grabbed that output too. here it is without keyring running, no problems

https://gist.github.com/bunder2015/e3bcf3e256d791351e69f48d4a17f041

sadly the patch doesn't apply cleanly, i guess i'll wait for gnome-keyring to push a new update. i don't really need it anyways.
_________________
Neddyseagoon wrote:
The problem with leaving is that you can only do it once and it reduces your influence.

banned from #gentoo since sept 2017
Back to top
View user's profile Send private message
quinn_jones
n00b
n00b


Joined: 14 Dec 2005
Posts: 50
Location: Wilmington, MA, US

PostPosted: Thu Jun 21, 2018 1:28 pm    Post subject: Reply with quote

Whissi wrote:
@ quinn_jones: Thanks for confirming. This is clearly a bug in VMware. Thanks to https://bugs.gentoo.org/658488#c3 I have seen captures showing RST packages from VMware when AF21/CS1 DSCP flags are used (this has nothing to do with OpenSSH, you can reproduce with everything.. you just need to set QoS marker to AF21/CS1 for something and VMware will kill that connection via RST package)

So if you have a valid VMware contract please file a bug against upstream. Would be nice if you could share ticket number.

This won't go away so you will need to keep IPQoS setting for now until VMware fixes their product.


Thank you very much, Whissi.

While my version of VMware player is licensed, I don't have a support contract. I've filed https://communities.vmware.com/message/2778248 lets see if it goes anywhere.

I guess I'll mark this issue as solved since I have a good work-around and it's not really Gentoo's problem -- though it could be solved by setting a different QoS in the default config, that's a kludge and is messy for anyone *not* running under VMware player.

Thanks again!

Quinn
Back to top
View user's profile Send private message
Whissi
Retired Dev
Retired Dev


Joined: 12 Jan 2011
Posts: 222

PostPosted: Thu Jun 21, 2018 1:43 pm    Post subject: Reply with quote

bunder wrote:
sorry i should have grabbed that output too. here it is without keyring running, no problems

https://gist.github.com/bunder2015/e3bcf3e256d791351e69f48d4a17f041

sadly the patch doesn't apply cleanly, i guess i'll wait for gnome-keyring to push a new update. i don't really need it anyways.
Thanks for confirming that the issue is gone when gnome-keyring is dead, i.e. not interfering with ssh. I filed a bug against Gentoo's gnome-keyring package, https://bugs.gentoo.org/show_bug.cgi?id=658646


@ Quinn: Thank you for posting at least in VMware's forum.
_________________
Regards,
Whissi
Back to top
View user's profile Send private message
bunder
Bodhisattva
Bodhisattva


Joined: 10 Apr 2004
Posts: 5934

PostPosted: Thu Jun 21, 2018 2:25 pm    Post subject: Reply with quote

great, thanks for the help.
_________________
Neddyseagoon wrote:
The problem with leaving is that you can only do it once and it reduces your influence.

banned from #gentoo since sept 2017
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2569

PostPosted: Thu Jun 21, 2018 2:42 pm    Post subject: Reply with quote

For those of you who are not yet fixed, I had a problem similar to this a year or two back. Don't remember the details except the error from the OP sounds very much like what I remember on my system. The subsequent investigation led me elsewhere though.

I don't intend for this post to derail the main discussion, just saying that for some of you it might actually be something else.

In my case, the remote hosts had had keys generated years before and the new local ssh version no longer supported those ciphers.

So you might consider regenerating ssh keys on one remote box to see if that fixes the issue.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum