[solved] "time exceeded in-transit"

Message

eqxro · Post by **eqxro** » Sun Mar 13, 2005 12:09 am

I've got a router with 2 ISPs, one LAN, resulting in a 3 NIC setup. NICs eth0, eth1 are the two ISPs and eth2 is the LAN. I let the users select their preffered gateway (I created a new routing table for each ISP, all identical but the default route). The problem is that if the system's default gateway is, let's say, through eth1, I can access the net from the router, but none of the users going out on eth1 have access to the internet. However, if one chooses the eth0 route, they get access to the internet. If I set the router's default gateway, the behavior is the same. eth1 can't access the internet from the LAN no matter what I try.

Wierd thing is it worked yesterday... I've got gentoo and shorewall with the wonder shaper trafic shaping tcstart script installed on the router. Here are my custom routes:

Code: Select all

Reboot shorewall # ip route show
192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1
82.76.28.0/24 dev eth0  proto kernel  scope link  src 82.76.28.129
10.0.252.0/22 dev eth1  proto kernel  scope link  src 10.0.253.42
127.0.0.0/8 dev lo  scope link
default via 10.0.255.1 dev eth1
Reboot shorewall # ip rule show
0:      from all lookup local
32759:  from 192.168.0.4 lookup T1
32760:  from 192.168.0.3 lookup T2
32762:  from 192.168.0.2 lookup T2
32766:  from all lookup main
32767:  from all lookup default
Reboot shorewall # ip route show table T1
192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1
82.76.28.0/24 dev eth0  proto kernel  scope link  src 82.76.28.129
10.0.252.0/22 dev eth1  proto kernel  scope link  src 10.0.253.42
127.0.0.0/8 dev lo  scope link
default via 82.76.28.1 dev eth0
Reboot shorewall # ip route show table T2
192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1
82.76.28.0/24 dev eth0  proto kernel  scope link  src 82.76.28.129
10.0.252.0/22 dev eth1  proto kernel  scope link  src 10.0.253.42
127.0.0.0/8 dev lo  scope link
default via 10.0.255.1 dev eth1

If I try pinging the DNS server from the LAN, I get:

Code: Select all

01:52:41.412056 IP (tos 0x10, ttl 127, id 6, offset 0, flags [DF], length: 84) localhost > thorin.mediasat.ro: icmp 64: echo request seq 7
01:52:41.435962 IP (tos 0x10, ttl   1, id 30343, offset 0, flags [none], length: 84) thorin.mediasat.ro > localhost: icmp 64: echo reply seq 7
01:52:41.436081 IP (tos 0xd0, ttl  64, id 38011, offset 0, flags [none], length: 112) localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit for IP (tos 0x10, ttl   1, id 30343, offset 0, flags [none], length: 84) thorin.mediasat.ro > localhost: icmp 64: echo reply seq 7

eqxro · Post by **eqxro** » Sun Mar 13, 2005 11:00 pm

Some more info, from tethereal:

Code: Select all

################################# PING send
Frame 18 (98 bytes on wire, 98 bytes captured)
    Arrival Time: Mar 14, 2005 00:55:08.632890000
    Time delta from previous packet: 0.996462000 seconds
    Time since reference or first frame: 4.998961000 seconds
    Frame Number: 18
    Packet Length: 98 bytes
    Capture Length: 98 bytes
    Protocols in frame: eth:ip:icmp:data
Ethernet II, Src: 4c:00:10:3a:a9:8f, Dst: 00:d0:b7:51:1b:cf
    Destination: 00:d0:b7:51:1b:cf (Intel_51:1b:cf)
    Source: 4c:00:10:3a:a9:8f (4c:00:10:3a:a9:8f)
    Type: IP (0x0800)
Internet Protocol, Src Addr: 10.0.253.42 (10.0.253.42), Dst Addr: 193.231.169.2 (193.231.169.2)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
        0000 00.. = Differentiated Services Codepoint: Default (0x00)
        .... ..0. = ECN-Capable Transport (ECT): 0
        .... ...0 = ECN-CE: 0
    Total Length: 84
    Identification: 0x00c7 (199)
    Flags: 0x04 (Don't Fragment)
        0... = Reserved bit: Not set
        .1.. = Don't fragment: Set
        ..0. = More fragments: Not set
    Fragment offset: 0
    Time to live: 63
    Protocol: ICMP (0x01)
    Header checksum: 0xc8cd (correct)
    Source: 10.0.253.42 (10.0.253.42)
    Destination: 193.231.169.2 (193.231.169.2)
Internet Control Message Protocol
    Type: 8 (Echo (ping) request)
    Code: 0
    Checksum: 0xb001 (correct)
    Identifier: 0xa822
    Sequence number: 0x00c8
    Data (56 bytes)

0000  9d c3 34 42 00 00 00 00 00 3b 0e 00 00 00 00 00   ..4B.....;......
0010  10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f   ................
0020  20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./
0030  30 31 32 33 34 35 36 37                           01234567

################################# PING reply
Frame 19 (98 bytes on wire, 98 bytes captured)
    Arrival Time: Mar 14, 2005 00:55:08.634872000
    Time delta from previous packet: 0.001982000 seconds
    Time since reference or first frame: 5.000943000 seconds
    Frame Number: 19
    Packet Length: 98 bytes
    Capture Length: 98 bytes
    Protocols in frame: eth:ip:icmp:data
Ethernet II, Src: 00:d0:b7:51:1b:cf, Dst: 4c:00:10:3a:a9:8f
    Destination: 4c:00:10:3a:a9:8f (4c:00:10:3a:a9:8f)
    Source: 00:d0:b7:51:1b:cf (Intel_51:1b:cf)
    Type: IP (0x0800)
Internet Protocol, Src Addr: 193.231.169.2 (193.231.169.2), Dst Addr: 10.0.253.42 (10.0.253.42)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
        0000 00.. = Differentiated Services Codepoint: Default (0x00)
        .... ..0. = ECN-Capable Transport (ECT): 0
        .... ...0 = ECN-CE: 0
    Total Length: 84
    Identification: 0x3445 (13381)
    Flags: 0x00
        0... = Reserved bit: Not set
        .0.. = Don't fragment: Not set
        ..0. = More fragments: Not set
    Fragment offset: 0
    Time to live: 1
    Protocol: ICMP (0x01)
    Header checksum: 0x1350 (correct)
    Source: 193.231.169.2 (193.231.169.2)
    Destination: 10.0.253.42 (10.0.253.42)
Internet Control Message Protocol
    Type: 0 (Echo (ping) reply)
    Code: 0
    Checksum: 0xb801 (correct)
    Identifier: 0xa822
    Sequence number: 0x00c8
    Data (56 bytes)

0000  9d c3 34 42 00 00 00 00 00 3b 0e 00 00 00 00 00   ..4B.....;......
0010  10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f   ................
0020  20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./
0030  30 31 32 33 34 35 36 37                           01234567

################################# PING time exceeded in-transit
Frame 20 (126 bytes on wire, 126 bytes captured)
    Arrival Time: Mar 14, 2005 00:55:08.635070000
    Time delta from previous packet: 0.000198000 seconds
    Time since reference or first frame: 5.001141000 seconds
    Frame Number: 20
    Packet Length: 126 bytes
    Capture Length: 126 bytes
    Protocols in frame: eth:ip:icmp:ip:icmp:data
Ethernet II, Src: 4c:00:10:3a:a9:8f, Dst: 00:d0:b7:51:1b:cf
    Destination: 00:d0:b7:51:1b:cf (Intel_51:1b:cf)
    Source: 4c:00:10:3a:a9:8f (4c:00:10:3a:a9:8f)
    Type: IP (0x0800)
Internet Protocol, Src Addr: 10.0.253.42 (10.0.253.42), Dst Addr: 193.231.169.2 (193.231.169.2)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0xc0 (DSCP 0x30: Class Selector 6; ECN: 0x00)
        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (0x30)
        .... ..0. = ECN-Capable Transport (ECT): 0
        .... ...0 = ECN-CE: 0
    Total Length: 112
    Identification: 0x606b (24683)
    Flags: 0x00
        0... = Reserved bit: Not set
        .0.. = Don't fragment: Not set
        ..0. = More fragments: Not set
    Fragment offset: 0
    Time to live: 64
    Protocol: ICMP (0x01)
    Header checksum: 0xa74d (correct)
    Source: 10.0.253.42 (10.0.253.42)
    Destination: 193.231.169.2 (193.231.169.2)
Internet Control Message Protocol
    Type: 11 (Time-to-live exceeded)
    Code: 0 (Time to live exceeded in transit)
    Checksum: 0xf4ff (correct)
    Internet Protocol, Src Addr: 193.231.169.2 (193.231.169.2), Dst Addr: 10.0.253.42 (10.0.253.42)
        Version: 4
        Header length: 20 bytes
        Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
            0000 00.. = Differentiated Services Codepoint: Default (0x00)
            .... ..0. = ECN-Capable Transport (ECT): 0
            .... ...0 = ECN-CE: 0
        Total Length: 84
        Identification: 0x3445 (13381)
        Flags: 0x00
            0... = Reserved bit: Not set
            .0.. = Don't fragment: Not set
            ..0. = More fragments: Not set
        Fragment offset: 0
        Time to live: 1
        Protocol: ICMP (0x01)
        Header checksum: 0x1350 (correct)
        Source: 193.231.169.2 (193.231.169.2)
        Destination: 10.0.253.42 (10.0.253.42)
    Internet Control Message Protocol
        Type: 0 (Echo (ping) reply)
        Code: 0
        Checksum: 0xb801 (correct)
        Identifier: 0xa822
        Sequence number: 0x00c8
        Data (56 bytes)

0000  9d c3 34 42 00 00 00 00 00 3b 0e 00 00 00 00 00   ..4B.....;......
0010  10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f   ................
0020  20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./
0030  30 31 32 33 34 35 36 37                           01234567

eqxro · Post by **eqxro** » Mon Mar 14, 2005 6:23 pm

Okay, I solved this, it seems my ISP sent back al the packets with TTL=1 and they couldn't be forwarded anymore from my router (it would die on the server). I had to patch my kernel to be able to do something like iptables -t mangle -A PREROUTING -i eth1 -j TTL --ttl-set 64. The patch is patch-o-matic-ng, the TTL part only.