jesnow
l33t
Joined: 26 Apr 2006
Posts: 857
|
 Posted: Tue Apr 30, 2024 5:37 pm Post subject: wireguard stopped working (again) [not solved] |
 |
|
Yet again, wireguard fails silently. At least the debug chain is less frustrating because I know it's probably wireguard's fault.
I sit down at my computer, and my remote mounts aren't available. OK. So I ssh through the wg tunnel, nothing, wg is broken. So when did this happen? Four days ago at midnight. Makes sense, I haven't used the machine since that (WFH). My previous experiences (two debug sessions) with this problem are here:
https://forums.gentoo.org/viewtopic-t-1167259-highlight-.html
https://forums.gentoo.org/viewtopic-t-1166203-highlight-.html
Only this time I haven't rebuilt the kernel. It just stopped working four days ago. I can see this from the output of wg (last handshake) and this:
| Code: |
vanaert jesnow # grep nfs: /var/log/messages
Apr 26 00:55:20 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 00:55:49 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 00:57:48 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 00:58:19 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 00:59:46 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 00:59:46 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 01:01:59 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 01:01:59 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 01:04:12 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 01:06:26 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 03:57:16 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 03:59:29 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 03:59:29 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 12:51:58 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 12:51:58 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
Apr 26 12:54:11 vanaert kernel: nfs: server 10.0.17.1 not responding, still trying
|
NFS is thankfully doing me the favor of telling me that it had a connection that suddenly stopped responding.
I restart nfs and wireguard. Nothing. But also no error messages, just nothing. I really wish that either of them when they are told there should be an interface there or a remote mount that they should be seeing, that it's a problem and they should write to the syslog when there is nothing there. I get *endless* chatter in /var/log/messages from dbus and networkmanager, and other super chatty daemons, to the point where it's hard to scan the logs for problems because there's so much junk filling it up.
But I can't connect to my remotes, because the wireguard interface is dead and wireguard is "OK this is fine". And I have no idea that it's not working and no way of knowing why it's not working.
So as in the previous threads I re-installed net-vpn/wireguard-tools, this rebuilds the key signatures from the wg0.conf. And rebooted. Nothing.
Anybody with real insight into this problem please let me know. I'm finding it frustrating how often wireguard fails silently on me. Silent fail is the worst. I have other work to do that isn't this, and I'm glad I have a backup machine.
Update:
With debugging turned on, it's the same symptoms as last time.
| Code: |
Apr 30 19:39:34 vanaert kernel: wireguard: wg0: Handshake for peer 1 (104.176.81.55:51820) did not complete after 5 seconds, retrying (try 6)
Apr 30 19:39:34 vanaert kernel: wireguard: wg0: Sending handshake initiation to peer 1 (104.176.81.55:51820)
Apr 30 19:39:39 vanaert kernel: wireguard: wg0: Handshake for peer 1 (104.176.81.55:51820) did not complete after 5 seconds, retrying (try 7)
Apr 30 19:39:39 vanaert kernel: wireguard: wg0: Sending handshake initiation to peer 1 (104.176.81.55:51820)
Apr 30 19:39:44 vanaert kernel: wireguard: wg0: Handshake for peer 1 (104.176.81.55:51820) did not complete after 5 seconds, retrying (try 8)
Apr 30 19:39:44 vanaert kernel: wireguard: wg0: Sending handshake initiation to peer 1 (104.176.81.55:51820)
Apr 30 19:39:50 vanaert kernel: wireguard: wg0: Handshake for peer 1 (104.176.81.55:51820) did not complete after 5 seconds, retrying (try 9)
Apr 30 19:39:50 vanaert kernel: wireguard: wg0: Sending handshake initiation to peer 1 (104.176.81.55:51820)
Apr 30 19:39:55 vanaert kernel: wireguard: wg0: Handshake for peer 1 (104.176.81.55:51820) did not complete after 5 seconds, retrying (try 2)
Apr 30 19:39:55 vanaert kernel: wireguard: wg0: Sending handshake initiation to peer 1 (104.176.81.55:51820)
Apr 30 19:40:00 vanaert kernel: wireguard: wg0: Handshake for peer 1 (104.176.81.55:51820) did not complete after 5 seconds, retrying (try 3)
Apr 30 19:40:00 vanaert kernel: wireguard: wg0: Sending handshake initiation to peer 1 (104.176.81.55:51820)
Apr 30 19:40:05 vanaert kernel: wireguard: wg0: Sending handshake initiation to peer 1 (104.176.81.55:51820)
...
Apr 30 19:40:52 vanaert mount[5447]: mount to NFS server '10.0.17.1' failed: timed out, retrying
|
Ping also obviously does not work, nor ssh through the non-existent tunnel connection.
But regular ssh login to that server works fine:
| Code: |
jesnow@vanaert ~ $ ssh merckx
Last login: Tue Apr 30 19:45:46 2024 from 130.39.188.91
jesnow@merckx ~ $
|
Showing that the connection is just fine end to end. Just wireguard has yet again failed silently. Last time it was a problem with the keys needing to be ported between the old and new kernel, there are multiple ways to do this (see the chain of posts), and I tried them all. The system is up to date. My backup system (nearly identical setup just much slower) is working fine: wg, nfs, the works.
Kind of at a loss.
[another update] I have a workaround that doesn't use wireguard -- I can ssh into my server and start a forward tunnel and connect to samba through it. It's about as fast, but I'm still at a loss why wireguard would simply and without any error message would refuse to work on this machine.
Cheers,
Jon. |
|
Networking & Security
