I could use some help with this masquerading issue [SOLVED]

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

I am replacing my main router/desktop machine. It is on a cable modem. The original machine masquerades my outbound traffic, has an ipsec tunnel to my office network, runs bind, ntp, dhcpd for my internal network and works fine albeit slowly. I have a new machine to replace it. Configs are all the same. For testing I have dropped the ipsec and reduced my iptables script to:

Hu · Moderator Joined: 06 Mar 2007 Posts: 21635

This sounds like an MTU/MSS issue. Usually, that happens when fragmentation is needed and the ICMP Fragmentation Needed message is lost. However, there could be an upstream device blocking the ICMP-FN message and your old machine happened to have a low enough MTU that things worked anyway. For both the old and new machines, what is the output of /sbin/ip addr show, when plugged in and serving LAN requests to the Internet?

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

Howdy. Thanks for the quick reply.

Here's the old machine in place and working:

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

Okay, here is from the new machine:

Hu · Moderator Joined: 06 Mar 2007 Posts: 21635

An MTU of 576 is obnoxiously low, but at least the old and new machines are consistent. When you tested with the old machine, did you use the same set of rules as shown in the first post? A network with low MTU may need to clamp the MSS of internal machines, if they expect a full MTU of 1500.

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

Well shoot... It's working now.

Yea, the configs and addressing were the same on both boxes while testing. The low MTU comes from this option in my dhcpcd.conf:

Hu · Moderator Joined: 06 Mar 2007 Posts: 21635

Your dhcpcd is setting an MTU of 576 because the upstream DHCP server tells it to do that. The interesting question is whether the upstream server should be doing that. I have seen some DHCP servers, typically from consumer level ISPs, advertise an MTU of 576 when they should not do so. Unfortunately, advertising a wrong MTU, whether too low or too high, causes undesirable behavior. Similarly, overriding the advertised MTU to a value that is too low or too high can cause that same undesirable behavior. In both cases, if the interface MTU is lower than the link MTU, then you fragment packets unnecessarily, which hurts performance by generating needless copies of the control headers. On the other hand, if the interface MTU is higher than the link MTU, then you will generate packets which get rejected, forcing you to fragment them down to the right size. This also hurts performance, since it means you send a packet, get it rejected, then resend it in proper form. If the interface MTU were set properly, you would generate them with the right size on the first pass.

In the really bad case, using too high an MTU can cause connections to hang, exactly as you originally saw, because the generated packet is rejected, but some node along the path drops the notification that the sender needs to fragment and retry. The sender is then left hanging. If it retries at all (as TCP does), it will likely retry with the exact same segment size, causing another drop. Linux provides some mechanisms to compensate for this when doing NAT, but the proper solution for this situation is to find the device that is dropping ICMP-FN packets and replace it with one that works properly.