All ssh based large file transfers stall

Message

JanErik · Post by **JanErik** » Thu May 10, 2012 7:14 pm

I have a problem on my file server that all ssh based (scp, rsync, rdiff-backup) file transfers stall on large files. It does not work in either direction.
Over NFS transfers works fine.

Seems it stopped working after I updated the machine last weekend after running a few months without update.

I have tried a different NIC, cable and switch port. Have run revdep-rebuild and python-updater. Also re-emerged libssh and openssh without any improvement.

swimmer · Post by **swimmer** » Thu May 10, 2012 11:35 pm

Same here - I even had to switch my rdiff-backup config from ssh to NFS :-/

The only visible change I could see was the switch from openssh 5.9_p1-r4 to 6.0_p1 .... but there was only *one* change in the sshd_config so far ...

Code: Select all

# Allow client to pass locale environment variables #367017
AcceptEnv LANG LC_*

PaulBredbury · Post by **PaulBredbury** » Fri May 11, 2012 4:00 am

Try openssl 1.0.0j instead. Then recompile openssh.

The 1.0.1 series seems buggy.

furanku · Post by **furanku** » Sat May 12, 2012 7:54 am

Same here. Going back to openssl-1.0.0j and reemerging/restarting openssh did not solve the issue.

enigma59 · Post by **enigma59** » Sat May 12, 2012 1:26 pm

I have the same issue doing something as simple as

Code: Select all

cat largefile | ssh localhost cat > another_large_file

stalls it if largefile is sufficiently large, I used 4mb. Downgrading to openssh-5.9_p1-r4 works correctly while openssh-6.0_p1 doesn't.

Edit: Just found this bug that has more discussion https://bugs.gentoo.org/show_bug.cgi?id=414401

JanErik · Post by **JanErik** » Sat May 12, 2012 5:05 pm

Downgrading to openssh-5.9_p1-r4 solved the problem.

Ant P. · Post by **Ant P.** » Mon May 14, 2012 2:35 pm

Found a less intrusive workaround from a post on gentoo-user: setting "TcpRcvBufPoll no" in sshd_config fixed it for me.

wazoo42 · Post by **wazoo42** » Mon May 14, 2012 7:30 pm

Ant P.s solution worked for my as well, while the different versions of openssl/openssh did not. One small note, sshd did not like "No" and I had to change it to "no".

Ant P. · Post by **Ant P.** » Mon May 14, 2012 11:59 pm

My bad, "no" is correct. Fixed.

swimmer · Post by **swimmer** » Tue May 15, 2012 8:29 am

Ant P. wrote:Found a less intrusive workaround from a post on gentoo-user: setting "TcpRcvBufPoll no" in sshd_config fixed it for me.

That's the solution indeed! (At least for me)

Thank you very much Ant P. for sharing

depontius · Post by **depontius** » Tue May 15, 2012 12:25 pm

Ant P. wrote:Found a less intrusive workaround from a post on gentoo-user: setting "TcpRcvBufPoll no" in sshd_config fixed it for me.

Is there a way to do this from the client side? I can certainly do this on those systems which I administer, but not on my employer's systems, which I don't.

Havin_it · Post by **Havin_it** » Tue Jun 05, 2012 11:54 am

depontius wrote:
Ant P. wrote:Found a less intrusive workaround from a post on gentoo-user: setting "TcpRcvBufPoll no" in sshd_config fixed it for me.
Is there a way to do this from the client side? I can certainly do this on those systems which I administer, but not on my employer's systems, which I don't.

Depends how you are using SSH, but I've found that invoking it with compression ("-C") prevents the hangs I've been seeing in most cases. For rsync the arg you need is -c, for unison it's -sshargs="-C".

I also found in my case that file-size is not an issue, but it appears that certain files would always hang at a certain point (according to progress displays in rsync and unison) which makes me think that some byte[-sequence] is the trigger. Anyone else who is doing multi-file transfers, I'd welcome your own findings on that one, here or in the [bug=414401]bug-report[/bug].

hydrapolic · Post by **hydrapolic** » Thu Jun 07, 2012 1:58 pm

Ant P. wrote:Found a less intrusive workaround from a post on gentoo-user: setting "TcpRcvBufPoll no" in sshd_config fixed it for me.

thanks, this fixed the problem

EDIT:
net-misc/openssh-6.0_p1-r1 fixes the issue, so no need to set "TcpRcvBufPoll no" anymore (bug 414401)

Thanks

Dan · Post by **Dan** » Wed Jun 13, 2012 7:40 pm

Thanks guys. I been googling "scp stalled", on and off for a few days and trying all kinds of sysctl settings. Thanks for the real solution!