wireguard configuration and performance issues [solved]

[jesnow]

Note: My strange network issue was a bad NIC, it wasn't anything to do with wg or ssh. I also figured out wireguard completely for my use case, that's way down this thread.

I'm debugging my strange network issue still. to recap, I have ~10Mb/s upload speeds to my server from work when it's GBE end to end all the way from my home server to my desktop at work. I can download stuff and run tunnels and such just fine, but saving or copying anything to the sever is sl-o-o-o-w. I previously thought a misconfig of openssh must be at fault, so I tested many different configuration and encryption options. they all came out about the same: ~1GBE down, 0.01 GBE up. I came to the conclusions that it's not an openssh misconfig. In fact, everything works just fine, just slowly in one direction.

viewtopic-t-1159589-highlight-.html

Today I'm going to set up wireguard and see if the suggestion of several commenters that my bizarre abuse of ssh to open multiple reverse tunnels on each machine must be at fault. My work machines are behind a firewall that allows no incoming connections. My home machine is behind my home firewall that has just one pinhole in it on an nonstandard port for ssh and now another one for wireguard. I have some questions scattered through this, and I may make mistakes, please let me know if I've got it wrong so I can fix it.

Wireguard setup

Wireguard is still relatively new, and while its configuration is not super complex, it's tricky and the documentation s*cks [edit: is still somewhat sparse]. Programmers like to document their work in completely tautological sentences. "The EndPoint keyword is used to specify the end point of the encabulator phlogiston. Be careful to specify only values in DERP format." Wait, what? I exaggerate. But that's how it feels like.

Here is the /etc/wireguard/wg0.conf on my server:

Code: Select all

[Interface]
Address = 10.0.2.1/24
#SaveConfig = true
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o enp2s0 -j MASQUERADE;
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o enp2s0 -j MASQUERADE;
ListenPort = 51820
PrivateKey = ********************************************=

[peer]
PublicKey = *********************************************=
AllowedIPs = 10.0.2.0/24

The key things to note that may save you some grief someday:

1) The examples in the wireguard and gentoo documentation are st*pid [edit: Not useful for the majority of use cases]. They just put in nonsense numbers that nobody would use in the real world. Way to confuse the hell out of any body coming into this cold.

2) [edited] CIDR IP addresses with a bit width are used in two different ways, but they have to match in a very specific way between both peers. this is super confusing, but below I try to disentangle it.

2.5) You have to be able to edit these config files using ssh usually on both machines. You need to be root on both machines usually.

3) Address [edited thanks to szatox] is the address the the local machine will use in the *remote* address space. It has nothing to do with anything happening on the local machine. The docs don't really tell you this. It's extra confusing that its got a CIDR width on it as if it were a range of addresses starting at your server's address. But that's not how it works. 10.0.2.1/24 is the specific address 10.0.2.1 in a 32-24=8 bit range of addresses starting from 10.0.2.0 to 10.0.2.254. Everybody remembers this, right? It's how most local networks work. The CIDR width and address range must match on both machines, but the remote machines must not use addresses that collide. "Address" is like in your sshd_config file "ListenAddress", (but that doesn't need a bit width or netmask) -- sshd usually only operates in your local net. Gee it would sure be nice if the config files for client server applications at least tried to use similar terminology.

4) ListenPort This is the equivalent of "Port" in sshd_config. Usually this is the outside port that is used on the public net. I guess it is a kind of consistency to label these two exactly backwards from each other. The default port is 51820, but if you don't specify it, it will be random. I don't understand how anybody would ever talk to a sever that's listening on a random port, but I guess if you're only connecting to a server peer and not ever listening, then having a random open wireguard port at least doesn't hurt anything.

5) PrivateKey and PublicKey are pretty self explanatory, you do a keygen and copy the files similarly as for passwordless login for ssh. It would be nice if there were a script like ssh-copy-id that would reach out to your intended server (remote peer) and exchange the public keys automagically.

6) The [Interface] section is your local machine and [peer] section is the remote host. Why can't they call that server and client? because I don't know why, the authors don't find it confusing, so it's all good. It's good that the sections are simple and contained in the same file. They wanted I guess to emphasize that all machines are created equal. How about [local] and [remote]? Because honestly your local machine is also a peer and the remote machine also has an interface. So don't be confused.

7) Lots of stuff gets saved persistently, whether you like it or not. There is a SaveConfig keyword: it is the road to hell. How many times did I stop the service, change wg0.conf and restart it only to have the daemon change it back. Maybe once you have a stable config turn it on. But for example if your server's public IP changes you are pretty hosed. I don't know how you use dynamic dns with wireguard, a problem for another day. There must be a way, but not if the config file is constantly overwritten.

8) EndPoint is where you can specify a public network name and port to connect to. But it looks like once it connects it saves that state forever. Its SaveConfig overwrote my DDNS name for my server with its ip address. After I deleted the endpoint it still kept working through a restart, but I really wish it didn't. Once "SaveConfig" was set to false things started to be come less frustrating.

9) AllowedIPs You don't get to specify the address of your remote peer. I repeat, you don't get to specify the address of your remote peer. Your remote peer tells *you* what its IP address in the private network is, which it specifies in its own [interface] section. You should specify the address *range* your remote host could be in. You might as well put the base of the range, but the way CIDR's work, you can use any address within the range and it has no effect. Unless you get it wrong, and the range your remote peer is using and the range you are using don't overlap. Then they won't talk to each other. So: the range specified by AllowedIPs and the range specified by the Address keyword on the peer server you're connecting to need to match. Mostly you will want to use a single ip address here with a "/32" netmask.

Otherwise if you follow the instructions it works out pretty well.

Wireguard Monitoring

It's really hard to tell what the hell wireguard is doing. It seems to "just work" or "just not work". The most useful command is:

Code: Select all

merckx /home/jesnow # wg
interface: wg0
  public key: ******************************************=
  private key: (hidden)
  listening port: 51820

peer: ***********************************************=
  endpoint: 130.39.190.4:51820
  allowed ips: 10.0.2.0/24
  latest handshake: 1 minute, 20 seconds ago
  transfer: 151.92 MiB received, 1.28 GiB sent

10) If you see a [peer] section in this output it means you have a peer connected to your server, and the two are talking to each other. They very helpfully don't allow you to see the hostname of the remote machine, that would be really helpful to know, you only get to see the public key. You don't get to know the remote machine's ip address either or whether its a server peer or a client peer. In this example the fact that the peer is specifying an endpoint shows that the remote machine contacted the local machine, not the other way around. It should really be telling me that the remote machine's hostname is "pogacar" and its ip address in the tunnel is 10.0.2.2. I don't actually know how you're supposed to figure those things out to (eg) ssh into the machine. I guess you need to have an independent way of getting into the remote machine to get its IP address the first time. You also can't see the ip address of your local machine in the vpn address range. How do you avoid address collision in the vpn address range? It seems like the address allocation is back to /etc/hosts days. I guess I prefer "keep it simple".

You can at least learn your actual ip address you're using from ifconfig. But how do you log into your remote peer to find out its ip if you don't have it?

I have to go eat now, but I will return and get to the point eventually. [edit, I'm back]

Wireguard vs ssh -R

11) After using wireguard only a short time, I'm quickly becoming a fan. Ssh is able to create forward and reverse tunnels and manage multiple connections and do port forwarding, but wg appears to be *made* to do this. I have my questions about how to manage resources in the vpn on wg, but overall it seems very fast and powerful. without showing all the data, it seems to have less network overhead on a fast network than ssh does. But the difference between 500Mb/second and 700Mb/s (I did a lot of tests with iperf), isn't really that noticeable. What *is* noticeable is in a wonky network like my workplace seems to have. they claim to have GBE (in fact the big servers have 40GBE) to the outside world, but while my desktop gets its rated 1GBE on the download, on the upload it's throttled (intentionally or not) to 10Mb/s. Like coax 10BaseT speeds, sometimes worse. Wireguard triples that to 30Mb/s compared to ssh, and that is really noticeable in real world applications.

12) Wg on gentoo is well integrated into the init system. So you can add the init to default via rc-update, and it will come up and be available after a reboot (due eg to a power outage). This is pretty hard to do with ssh tunnels. It is doable, but it's hard.

13) Wg solves the TCP over TCP bandwidth killer that ssh tunnels have. That may be the reason for my much better performance.

Cheers,
Jon.

[szatox]

Man, you ramble a lot. Like in: A LOT

1. Dunno, maybe ,it's been a whale, I don't remember
2. You can put whatever mask fits your use case, obviously. In a setup wg was designed for - full-mesh - /32 makes perfect sense
3. No. Address is the IP assigned to the virtual interface attached to the tunnel. Server will listen on ISP provided IP, get your frame of reference right
4. Listen port is the port used for maintaining the encrypted tunnel stretched over the public network. Wg does not listen to any traffic at all on your private side, so there is nothing to confuse this with. I now want to know the story behind this point.
5. When the only thing in your toolbox is a hammer, the whole world looks like a nail. Use ssh for remote shell, it works really well, you should try it
6. Because wireguard is not designed to run in client-server architecture. It's meant to run in full mesh. 2 peers connected with a single link is a very simple case of a full mesh.
7. Did you expect it to NOT save after you specifically requested it? I didn't know this keyword existed and let me tell you, my ignorance is bliss
8. Parsing error. Undefined variable "That line". Also, what is so puzzling about names being resolved when tunnel is formed? How often would you like to have names resolved?
9. Sure, go ahead, create a node with mask /0 and then then try to add a 3rd peer to your network. I'll wait.



> I have to go eat now, but I will return and get to the point eventually.I have to go eat now, but I will return and get to the point eventually.
Good. You're not yourself when you're hungry

[jesnow]

Thanks very much:

I'm learning to do this to replace ssh -R which I've been using for years to pierce the two firewalls between my office and home. That's a very mature code base and lots of documentation how to use it, pitched at various levels.

Man, you ramble a lot. Like in: A LOT

Sorry about that, if you find my words slow going, then don't read them. If you think this is rambling you should hear me explaining crystal symmetry.

1. Dunno, maybe ,it's been a whale, I don't remember

A whale? <smiling emoji>

But, precisely the point, it's people like you (no offense) writing the documentation who don't remember what it's like not to know this stuff. There are some good guides out there, but not very many because it seems (maybe like gentoo) there's a huge learning curve. Here at gentoo we have lots of patient people to help people up the learning curves, but still the documentation doesn't get you anywhere near a working system. What's funny is that so many people on the previous discussion I linked said "just use wireguard". The "just" is so often a red flag hiding something enormously complicated and difficult.

2. You can put whatever mask fits your use case, obviously. In a setup wg was designed for - full-mesh - /32 makes perfect sense

It's like saying "just speak German". If the documentation is confusing, or only written for people who already know it, it's a very difficult thing to do.

3. No. Address is the IP assigned to the virtual interface attached to the tunnel. Server will listen on ISP provided IP, get your frame of reference right :P

Precisely the crazy part. "Address" appears in the [Interface] section, but it tells the remote peer what address I the local peer will be using. It has nothing to do with the local interface.

4. Listen port is the port used for maintaining the encrypted tunnel stretched over the public network. Wg does not listen to any traffic at all on your private side, so there is nothing to confuse this with. I now want to know the story behind this point.

Exactly: the port somebody uses from outside to contact the local server. I will clarify this in my post. It seems that the having a random listen port is a feature. There's nothing wrong with features I don't understand, these things are esoteric. Maybe having a listening port known only to God serves a greater purpose. But most of the time the remote peer needs to know the port on the public address, so it has to be exactly specified on both sides of the connection.

5. When the only thing in your toolbox is a hammer, the whole world looks like a nail. Use ssh for remote shell, it works really well, you should try it :lol:

Everything works great for me at present with ssh. I am probably wasting my time with wireguard, which is extremely difficult and doesn't appear to solve any more of my problems. But I'm working at it. The only way to find out is to try. The main thing I need to do is keep all my data in one place that I own no matter where I am in the world, including behind the spiky firewall at work. Nowadays you're supposed to give all your data to someone and they keep it and rent it back to you, but I don't like that. So a peer vpn like wg would seem to be ideal for my needs.

6. Because wireguard is not designed to run in client-server architecture. It's meant to run in full mesh. 2 peers connected with a single link is a very simple case of a full mesh.

I understand that. But that just means (like smb/cifs) every machine is both client and server. I thought I would get two peers working with each other reliably and understand that first before putting all my machines in the wg network and ditching ssh tunnels. My idea was that it would be much faster using wireguard (which makes some sense) rather than ssh -R (which is non-ideal but works) to connect linux peers. My use case (which I think is pretty common) makes that kind of peering impossible -- I have a server at home with my data on it, and the remote peers need to connect to it to get at the files and other services. I also like to use the remote peers as distcc volunteers.

7. Did you expect it to NOT save after you specifically requested it? I didn't know this keyword existed and let me tell you, my ignorance is bliss :lol:

Well I'm glad you learned something too. Many of the "getting started with wireguard" docs I read prior to diving in said to use SaveConfig. Big mistake. There seem to be many ways of using wg, and one some people prefer seems to be giving wg commands on the command line and letting *wg* manage the config file. Some other services work that way too. But when the documentation is poor, that little nuance gets lost. A beginner shouldn't try to mix the approaches. What cost me a bunch of time was not even that, but the fact that when I stopped the daemon and edited the conf file, the daemon when restarted appeared to remember the old state anyway and revert to it. It seemed to save the (incorrect) AllowedIPs assigned to the public key of the peer no matter what I did with wg0.conf. It wasn't until I deleted all the information about the peer and cycled it that I could re-enter the remote peer public key and change the AllowedIPs.

8. Parsing error. Undefined variable "That line". Also, what is so puzzling about names being resolved when tunnel is formed? How often would you like to have names resolved?

I will edit my post. "that line" was the EndPoint keyword -- wg overwrote it with the raw ip address, which is no good, as I have to use DDClient to maintain that address. The public IP will change, it needs to be looked up new every time. I had to stop the service, delete all the peer information, then restart it before I could re-enter new AllowedIPs. It took a while for the effects of SaveConfig to wear off. But then it worked.

9. Sure, go ahead, create a node with mask /0 and then then try to add a 3rd peer to your network. I'll wait. :lol:

I'm not sure what part of that point you're reacting to. My complaint (and what they don't tell you): Your client peer connecting from beyond the firewall tells *you* what IP it's going to use on the your end. AllowedIPs only specifies whether wg will talk to it or not.

> I have to go eat now, but I will return and get to the point eventually.I have to go eat now, but I will return and get to the point eventually.
Good. You're not yourself when you're hungry :lol:

There's more to this but I ran out of time. I did get everything working, and did a bunch of performance testing. I will continue that post.

My question to you in the meantime:
1) Where can you find out on your wg server (or local peer if you will) what IP's in your vpn address space the remote peers are using? If I *do* something on the remote peer (and for example ssh to my reverse tunnel onto the remote peer, then ssh back through wg), I can log in, which records my wg IP in /var/log/messages. That seems like a pretty nuts way to have to do it. And there's a chicken-and-egg involved. I see wg is up, I see that it has a peer, but unless I happen to know (maybe by recognizing the public key?) I have no idea what ip address to use to forward ssh into it, and since I don't know which peer it is (unless I happen to recognize the PublicKey) I don't know what password to use.

2) How can I prevent address collision between wg peers? This is handled in a normal network by dhcp -- peers joining the network get their IP dynamically. Having to edit config files to allocate addresses seems very 1990's.

I understand that wg is conforming to the unix philosophy of only doing one thing and doing it well. Up to us to route our packets or negotiate our address space.

Cheers,
Jon.

[szatox]

Important things first:
1.
The best way to set up wireguard I've seen so far is wg-quick. Highly recommend. A 2+3*peers lines long config file contains all you need, and it can even be started as an instantiated service.

Code: Select all

[Interface]
Address = 10.0.255.235/24
PrivateKey = ******

[Peer]
# comments are supported too, you can use them at will
PublicKey = *******
Endpoint = my.always.online.vps:udp_port
PersistentKeepalive = 20
AllowedIPs = 10.0.255.1/32

Since this machine does not have a static IP, Endpoint is not defined in corresponding [peer] section on the VPS - it relies on keepalive instead, which forces the "client" to initiate and maintain the tunnel whether it has anything to send or not.

2. I do that by adding new peers at the end of the config file on my hub and assigning them IPs in descending order. Simple enough, and works fine for a small, personal star-shaped VPN where clients don't need to talk to each other.
Doing an actual 10-node mesh I just took advantage of ansible I was already using in that environment and had all the configs templated to keep stuff in sync.




Some other tips, leaving the docs aside, since I suck at writing it bad enough it's not worth even trying; I used to just walk my junior through the process of doing whatever new thing and have her do the papers for the other sysadmins.

3. The address thing. I believe it's used in at least 3 places. ACL AKA "node X is allowed to send me stuff with source address of Y" - this one is in the docs.
Adding route vie wireguard interfaces to system's routing table. This is the default behaviour of wg-quick which makes perfect sense in most use cases, but can be disable via a config file directive if it happens to be getting in your way.
Wireguard's internal routing. It's just a hunch, I haven't been looking for confirmation, but wireguard must encrypt your outgoing packet with 1 key and send it to 1 Endpoint. It just makes sense to use the same ACL table for this purpose.

4 and 6. Damn, I knew I missed something. I don't remember how it behaves when port is undefined, I think it used a static default, but a random source port kinda makes sense too: When one node has a static IP (vpn server) and the other is a mobile device (vpn client), you can make up for the missing information by enabling keepalive on the client. Server will receive client's hello and learn it's current IP and port this way. Also, client changing it's IP address does not interrupt the tunnel or tcp sessions inside, you will just see a few packets lost or connections lagging until the client sends a new hello from another location.
WG peer identity is tied to its public key and NOT the Endpoint, and UDP is connection-less anyway, so there isn't really anything to break within the transport layer. Very convenient.
While this is NOT what wg was designed for, it is probably in line with what you want for your use case.

Well, that should be enough for now, it's plenty already.

[jesnow]

Not a long reply: wg is really amazing. If it were just a *little* better documented it would make everything else obsolete in its wake. As it is it seems to be a tricky tool to get going, that then keeps working almost whether you like it or not.

Tonight's pointers for the beginner (ie me a couple weeks ago):

1) You need to be root on both ends of the tunnel. Obvious and not a problem for most people, but different from ssh which can do some primitive tunneling as user.
2) You need to have reliable communication to your remote peer, really reliable. So make sure passwordless ssh works all ways to all machines before you get started with wg. You'l be using the ssh link *a lot* to debug why wireguard didn't start up right.
3) Suddenly, the security and encryption are all done for you. Depending on what services you're running through this tunnel , they don't need to be encrypted. Like samba and NFS now finally are. As long as you trust the network you're tunneling into, that all doesn't matter. Time to bring back telnet!
4) The sparseness of the documentation is a boon to third party services that want to manage it for you (for a fee of course). I didn't realize how many vpn providers use wg as their underlying technology. A lot.
5) If you're using ddns to keep track of your public endpoint, you need to have that completely sorted out first or it's going to be a mess. because:
[*]a) If you misconfigure ddclient and hammer the checkip.dyndns.org server for long enoug, you will get banned and that starts a world of hurt.
[*]b) but on the good side many ISP's change your public ip address so rarely that it might as well be static.
[*]c) google domains can be pretty slow about propagating your changed ip address out to the world. Better write that "pseudo-static" ip address down when you go to a new place.
6) Once you get used to it the level of persistence is awesome.

Cheers,
Jon.

[jesnow]

Wireguard is really hard. It turns out getting a single tunnel going is just the beginning of the journey.

More soon.

[jesnow]

Worked wireguard configuration example for a star topology, with tips and tricks.

I now have everything figured out. I can only praise wireguard, it ruthlessly follows the unix philosophy of doing one thing and doing it extremely well. I do some complaining up the thread about wireguard configuration (yes, it's pretty hard), but in the end, if you pay attention and understand what the sometimes confusingly-named parameters do, you are all set.

Wireguard devs if you're at all listening: I wish "wg show" had a verbose mode that tells us more of what wireguard knows about its state, that would make troubleshooting a lot easier. I have a workaround for that terseness.

So to wrap up this thread:

My use case is pretty simple, I have a home server and machines out in the wide world need to connect to it to move files back and forth. I did not connect the private net to my local nets on either end. I don't need it that urgently.

My config files:

First up, the local server (file below): This is /etc/wireguard/wg0.conf on the server Merckx. "Address" defines the private address space as 10.0.17.0/24 -- which means that the peers it can talk to will have addresses like 10.0.17.x, in CIDR notation. But we also kill two birds with one stone by specifying the address this peer will have on the private network in the last digit, so it's 10.0.17.1/24 for address 1. If you look up the thread, you will see that I used a different network range before. Just by chance, that network range (10.0.2.0/24) is the exact same one that VirtualBox uses for its internal networking, and so that made it impossible to use wireguard to connect to my laptop if virtualbox was running. It is apprently not configurable on virtualbox on the mac, so I had to change my wireguard address space. The same will be true if you choose a private address range for the vpn that's the same as the one at the internet cafe you're using. It will all just not work and not tell you why. So learn from my mistakes: pick a weird private address space. 192.168.0.0/24 for example will not serve you well.

The "postup" and "postdown" commands poke holes in your local machine's firewall. My local machine is behind a router firewall that routes it packets for wireguard, so I don't run iptables. The public interface on my router sends packets for port <secret> to port 51820 on this machine. My local net is trustworthy enough. I have the PrivateKey there of course. It's a good idea to put the PublicKey of the local peer in the comments right next to it, because wireguard only knows public keys. A given machine's ID in principle is its public key and nothing else. Every machine has a private IP address too (that it chooses), but wg won't always tell you that, you have to set it yourself and know it.

Each peer has a section that lists its public key and its AllowedIPs. "AllowedIPs" is counterintuitive: For this use case it's *not* plural, and you do *not* want a range. The best thing to do is specify an ip address with a /32 netmask -- one single ip address, period. This *only* says what ip's are allowed, it's up to the peer side to actually talk on that IP address, or it will be ignored. It is critical that AllowedIPs ranges of the peers must not overlap. If they do, only the first machine connecting in that range will be able to talk, the second one that tries will be silently ignored (no log messages), nothing for that peer will work, and you won't know why. That discovery cost me an entire day of banging my head against the wall.

Put the network name for each peer in the comments, because they are much easier to remember than public keys. I'll talk about dns in a moment.

merckx# cat /etc/wireguard/wg0.conf:

Code: Select all

[Interface]
# Merckx file server
Address = 10.0.17.1/24
#SaveConfig = false  # do not use this, it works, but causes pain
#PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o enp2s0 -j MASQUERADE;
#PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o enp2s0 -j MASQUERADE;
ListenPort = 51820
PrivateKey = ***
# PublicKey = bFLH***

[peer]
# pogacar remote peer
PublicKey = ha8K****
AllowedIPs = 10.0.17.2/32   #single fixed private ip address

[peer]
# chair mac laptop remote peer
PublicKey = 9Mxm****
AllowedIPs = 10.0.17.3/32   #single fixed private ip address

[peer]
# Vanaert remote peer
PublicKey = RtHD****
AllowedIPs = 10.0.17.4/32   #single fixed private ip address

Next is the peer, pogacar /etc/wireguard/wg0.conf:

[interface] section: Notice that I went ahead and put the network name of the remote peer in the comment of its config file. Very handy for side by side editing so you can remember which config file is which. If you confuse them and put the wrong key in the wrong file you will regret it. Here in "Address" is where you specify what address this remote peer machine will speak on. It has to match the IP you put in AllowedIPs on the server peer above, or you will get nothing. Again, this machine is behind a firewall so I don't need iptables. Privatekey is necessary, obviously. Notice that I put the public key in a comment too even though it's not required. This is so I can make sure that the public key for this peer on the server peer exactly matches. I had started to just memorize the first four digits of all the public keys, but this is stupid. We don't need a ListenPort, but I put it anyway, in theory other machines can connect to this one, maybe for a mesh setup. While this machine is not allowed to receive connections originating from outside ever (how I have pleaded with them), I have a couple machines at this location that could mesh. It's for the future. Right now I have a star topology, period.

[peer] section. This machine has only one peer, the server "Merckx", so put that in the comment. His PublicKey is bFLH***, it's a good one to remember. I can go verify side by side that the public keys match because I put Merckx's publicKey in the comment in his /etc/wireguard/wg0.conf. The EndPoint is merckx's public IP address and ListenPort, the one I bought on goodle domains and set up with ddns to point to my home server. This is good practice but not necessary. You can use a raw IP, they don't change much for most home routers (I've had the same one for 3.5 years). PersistentKeepalive is good, it lets you know that the tunnel is up and running. For some reason the value 30 gives me a handshake time interval of 120 seconds, I don't know why. Who cares. AllowedIPs once again should specify a *single* ip address and /32 netmask that your server peer will speak on, and it has to match the one in the server peer's [interface] section, but here it should be /32 where the netmask had some other value (usually /24) on the server peer. This is explained exactly nowhere that I've found.

You use the exact same [peer] section on every peer! That makes it "easy".

Code: Select all

pogacar jesnow # cat /etc/wireguard/wg0.conf 
[Interface]
# Pogacar
Address = 10.0.17.2/32
#SaveConfig = true
#PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o en01 -j MASQUERADE;
#PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o en01 -j MASQUERADE;
ListenPort = 51820
PrivateKey = ****
# public key: ha8K****


[Peer]
#Merckx
PublicKey = bFLH****
AllowedIPs = 10.0.17.1/32
Endpoint = merckx.vesarius.net:51820
PersistentKeepalive = 30

Starting wireguard

Bring up the wireguard tunnel on both sides of the pipe. There are several ways to do this but the Gentoo way is to use openrc. Gentoo's package net-vpn/wireguard-tools installs a wg-quick init script in /etc/init.d/ -- that is different from the wg-quick command, which also works on the command line. Or the wg command, which if you feed it the right options also works. But to do it the gentoo way, link /etc/init.d/wg-quick to /etc/init.d/wg-quick.<tunnel name, usually wg0> like you do with ethernet interfaces, and call it with "/etc/init.d/wg-quick.wg0 start". Add it to a runlevel with "# rc-update add wg-quick.wg0 default" if you like. If all goes well you will see something like this:

Code: Select all

wg-quick.wg0              | * Starting WireGuard via wg-quick(8) for wg0 ...
wg-quick.wg0              |[#] ip link add wg0 type wireguard
wg-quick.wg0              |[#] wg setconf wg0 /dev/fd/63
wg-quick.wg0              |[#] ip -4 address add 10.0.17.2/32 dev wg0
wg-quick.wg0              |[#] ip link set mtu 1420 up dev wg0
wg-quick.wg0              |[#] ip -4 route add 10.0.17.1/32 dev wg0                                                             [ ok ]
pogacar jesnow # 

If the link is up on both sides you can see that too:

Code: Select all

pogacar jesnow # wg show
interface: wg0
  public key: ha8K****
  private key: (hidden)
  listening port: 51820

peer: bFLH****
  endpoint: 104.176.81.55:51820
  allowed ips: 10.0.17.1/32
  latest handshake: 56 seconds ago
  transfer: 23.91 KiB received, 24.26 KiB sent
  persistent keepalive: every 30 seconds

You know that you are in, because the lines "latest handshake" and "transfer" appear. That means something good has happened. Wireguard is not chatty, so if you haven't specified a keepalive it will open a perfectly good tunnel, but not show the handshake or transfer lines until data is sent. Wg shows you the remote ip of your server peer, but you can never be sure of the ip address that your machine is actually speaking on unless you were watching when openrc brought up the interface. *You* know what you put in the config file, but there is no way to verify it on your local machine. You have to go back to the peer (in this case my peer server merckx) to find that out. There is no trace of the tunnel appearing in /var/log/messages.

So back over to the server machine merckx (which was also has wireguard started with the init script). I now have several peers running talking to merckx.

Code: Select all

merckx /home/jesnow # wg
interface: wg0
  public key: bFLH****
  private key: (hidden)
  listening port: 51820

peer: ha8K****
  endpoint: 130.39.190.4:51820
  allowed ips: 10.0.17.2/32
  latest handshake: 11 seconds ago
  transfer: 386.77 KiB received, 383.27 KiB sent

peer: RtHD****
  endpoint: 130.39.190.142:51820
  allowed ips: 10.0.17.4/32
  latest handshake: 1 minute, 31 seconds ago
  transfer: 8.39 KiB received, 4.57 KiB sent

peer: 9Mxm****
  allowed ips: 10.0.17.3/32

Here it gets a little confusing because wg doesn't give you much means of identifying the peers. No hostname, no nickname, only their PublicKey and ip. But not *my* (Merckx's) private ip address. I have to just know that. At least by matching the public key ha8K**** I can see that pogacar indeed as intended is speaking on IP 10.0.17.2/32, and that it is handshaking. No so much for the peer 9Mxm****, you have to just somehow know what that is (because you put comments in your /etc/wireguard/wg0.conf!) so this is my laptop, and I know it's hibernating right now. Wg knows it hasn't heard from my laptop in a while. How long is a while? When was its last successful handshake? Wireguard doesn't say.

So now everything looks good and all machines can ping merckx, and merckx can ping them back. And run ssh, iperf, or any other service. Only with raw ip's. If you want to use names you have to configure that.

DNS over wireguard the simple way

So you can use bind, or dnsmasq. These will work and wg has a means of pointing dns queries to them. I haven't set any of that up because I only have 5 machines that don't all need to talk to each other. Or at least I can use merckx as a jump host. All my machines still can ssh to merckx the normal way through the merckx's public ip (for remote machines). It would be dumb to mess that up -- I think you need to be able to use both methods side by side for now. So I put and entry for "merckxw" in /etc/hosts and ~.ssh/config on all the remote machines. That way I can "ssh merckx" (which starts a lot of tunnels) and "ssh merckxw" which goes the same place through the wg0 tunnel without starting up any ssh tunnels of its own. This does the job for now.

Code: Select all

pogacar jesnow # cat /etc/hosts
#------------------------------------------------------------------------------
# Modified Calculate Utilities 3.7.2.13
# Processed template files:
# /var/db/repos/calculate/profiles/templates/3.6/3_ac_install_live/1-merge/sys-apps/baselayout/hosts
# To modify this file, create a /etc/hosts.clt template.
#------------------------------------------------------------------------------
127.0.0.1       pogacar pogacar.lsu.edu localhost
130.39.190.142  vanaert vanaert.lsu.edu

104.176.81.55   merckx.vesarius.net merckx
10.0.17.1       merckxw
pogacar jesnow # 

Samba over wireguard

Samba is a perfectly good networking solution for linux machines to talk to each other, and it works great over wireguard tunnels. Nfs I assume will work too, I intend to do some testing of which is faster for me and tuning them. but not now. Here is the mount command I use for samba:

Code: Select all

pogacar jesnow # mount -t cifs //10.0.2.1/jesnow /mnt/test -o credentials=/root/smb-merckx/.cred,vers=3.11,uid=jesnow,gid=users

Just for the record I use a nearly identical mount command for samba over an ssh tunnel:

Code: Select all

pogacar jesnow # mount -t cifs //localhost/jesnow /mnt/merckx-jesnow -o port=44445,credentials=/root/smb-merckx/.cred,vers=3.11,uid=jesnow,gid=users

That's it for now.

Thanks to everyone who helped me get this far. It wasn't that easy a journey. BUT you don't have to have a commercial vpn service just to have all your machines wherever they are talk to each other securely. Wireguard does a very good job of that.

Cheers,
Jon.