gretap Ethernet tunnels not fragment packets + other issues

[alex6z]

There is an obscure feature in Linux and iproute2 called gretap that lets you create a virtual Ethernet tunnel, over at least IPv4. Unlike GRE which encapsulates packets and sends them through a point-to-point link, but cannot be bridged in an Ethernet bridge, gretap encapsulates the entire Ethernet frame which is 18 bytes (or 22 with VLANs) on top of the packet that it contains. This type of tunnel can be bridged and added to a Linux Ethernet bridge. It's a really easy way to connect Ethernet networks through the Internet, or at least it should be easy.

There are a lot of problems.

The way I see it, an Ethernet tunnel should not inspect the contents of the packets other than the Ethernet frame, it should allow a 1500 byte MTU size in order to be compatible with standard Ethernet networks, it should fragment packets or allow fragmentation as necessary to maintain that 1500 byte MTU, and it should have nothing to do with ICMP or any other layer 3 protocols. Ethernet networks don't support fragmentation or error reporting or making decisions based on the contents of Ethernet frame payloads, and neither should the tunnel.

1) My 1st issue is that the default gretap interface MTU is 1462. But I have to lower the MTU to 1424 or more or there are major problems with TCP connections not working. Sending packets that are larger than 1386 bytes with ping -s 1358 results in the packets being dropped. This value was 1424 earlier today.

2) The gretap interface appears to set the TTL of generated packets to the TTL of encapsulated Ethernet+IPv4 frames. It should not do this! traceroute now shows 14 empty hops from arriving at my tunnel provider to the other end of the tunnel. Basically, packets must have 14 TTL hops left when they reach my tunnel server or they won't make it through. This is an Ethernet tunnel! There are no routers, or hops, or TTL values at all!

3) It will not fragment packets on its own, and as the links below explain, even though there is a do not fragment nopmtudisc option set, it wall turn it back on again and so the IP layer won't fragment it either! The gretap device should just generate packets that are larger than 1500 bytes and the IP layer (which allows 65535 byte packets) would just generate fragmented IP packets. I know TCP/IP has path MTU discovery and the don't fragment bit exists for a reason, but is it good to relay on that and there should be a way to force fragmentation anyway.

This article talks a lot about the Don't Fragment flag, and how the packets generated by the tunnel are having the DF flag set based on inspecting the contents of the packet, which it shouldn't be doing right?
http://backreference.org/2013/07/23/gre ... d-nfqueue/

This describes a bug about gretap not fragmenting packets. Apparently it hasn't been fixed.
https://bugzilla.kernel.org/show_bug.cgi?id=14837


It looks like it may be possible to get it working after going to a whole lot of trouble, but is this the best way? It seems like the gretap driver needs to be overhauled. A kernel patch would be really nice! Has anyone else had experience with these issues?

Thanks

[AngelKnight]

If you're still doing this, save yourself some trouble and use OpenVPN to bridge two segments together. By default it will emulate a 1500MTU-clean link by conducting fragmentation-and-reassembly on its own.

[alex6z]

I've read about another program called vtun. Do you know if it can is also a good choice?