View previous topic :: View next topic |
Author |
Message |
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Tue Oct 09, 2012 7:58 pm Post subject: [FIXED] How to route the router's traffic |
|
|
Hard to get title... well this is the case:
I have a linux router (gentoo) with 2 gateways WAN1 and WAN2.
The same machine have a LAN where all my machines are connected with UPNP, DHCP and such.
As a router it works perfectly, I'm using this method:
LAN user wants to go to 1.2.3.4
iptables mangle marks everything going to 1.2.3.4 as mark1
ip route knows it should go thru WAN1 (using table1)
all further packets to 1.2.3.4 go thru WAN1
LAN user wants to go to 33.33.33.33
iptables mangle marks everything going to 33.33.33.33 as mark2
ip route knows it should go thru WAN2 (using table2)
all further packets to 33.33.33.33 go thru WAN2
But... if i visit 33.33.33.33 from the router itself (ping, wget, etc), iptables doesn't do anything and it uses the default gateway which is WAN1 instead of going out thru WAN2 like it should have.
How can I force localhost to decide where to go using iptables mangle instead of using the default gateway directly? _________________ Practice Makes Perfect
Last edited by kuteninja on Wed Oct 10, 2012 11:47 pm; edited 1 time in total |
|
Back to top |
|
|
papahuhn l33t
Joined: 06 Sep 2004 Posts: 626
|
Posted: Tue Oct 09, 2012 8:57 pm Post subject: |
|
|
You need to mangle in OUTPUT for that. _________________ Death by snoo-snoo! |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Tue Oct 09, 2012 8:58 pm Post subject: |
|
|
papahuhn wrote: | You need to mangle in OUTPUT for that. |
Oh, so the local traffic from the router goes directly thru mangle->output and the LAN traffic goes thru mangle->prerouting or I should use only output? _________________ Practice Makes Perfect |
|
Back to top |
|
|
papahuhn l33t
Joined: 06 Sep 2004 Posts: 626
|
Posted: Tue Oct 09, 2012 8:59 pm Post subject: |
|
|
Use both. _________________ Death by snoo-snoo! |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Tue Oct 09, 2012 9:05 pm Post subject: |
|
|
papahuhn wrote: | Use both. |
Nope, still not working.
I've setup the same connmark rules on output and I'm still going out thru the default gateway instead of the iptables rule. _________________ Practice Makes Perfect |
|
Back to top |
|
|
papahuhn l33t
Joined: 06 Sep 2004 Posts: 626
|
Posted: Tue Oct 09, 2012 9:15 pm Post subject: |
|
|
Do you only use CONNMARK without packet MARK? The latter is actually relevant for policy routing. _________________ Death by snoo-snoo! |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Tue Oct 09, 2012 9:45 pm Post subject: |
|
|
Let's make it easier, here's my iptables mangle setup:
CONNMARK1 = WAN1
CONNMARK2 = WAN2
Code: | -A PREROUTING -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A PREROUTING -d 192.168.0.0/16 -j RETURN
-A PREROUTING -d 10.0.0.0/24 -j RETURN
-A PREROUTING -d 190.2.0.0/24 -m state --state NEW -j CONNMARK1
-A PREROUTING -d 66.220.158.0/24 -m state --state NEW -j CONNMARK2
-A PREROUTING -m mark ! --mark 0x1 -m state --state NEW -j CONNMARK2
-A PREROUTING -m mark --mark 0x0 -m state --state NEW -j CONNMARK2
-A OUTPUT -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A OUTPUT -d 192.168.0.0/16 -j RETURN
-A OUTPUT -d 10.0.0.0/24 -j RETURN
-A OUTPUT -d 190.2.0.0/24 -m state --state NEW -j CONNMARK1
-A OUTPUT -d 66.220.158.0/24 -m state --state NEW -j CONNMARK2
-A OUTPUT -m mark ! --mark 0x1 -m state --state NEW -j CONNMARK2
-A OUTPUT -m mark --mark 0x0 -m state --state NEW -j CONNMARK2
-A CONNMARK1 -j MARK --set-xmark 0x1/0xffffffff
-A CONNMARK1 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A CONNMARK2 -j MARK --set-xmark 0x2/0xffffffff
-A CONNMARK2 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff |
Let's say I visit 190.2.0.1 from my computer on the LAN, I go thru WAN1, then if I visit 66.220.158.1 I go thru WAN2.
I do the same on the router and I allways go thru WAN1 since it's the default gateway on route table main when I should use WAN1 or WAN2 depending on what iptables says. _________________ Practice Makes Perfect |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Wed Oct 10, 2012 8:29 pm Post subject: |
|
|
Sad bump
I couldn't find what's wrong.
I tried to remove the default gateway on the main table; the LAN kept having internet access but my router can't ping or browse by itself. _________________ Practice Makes Perfect |
|
Back to top |
|
|
papahuhn l33t
Joined: 06 Sep 2004 Posts: 626
|
Posted: Wed Oct 10, 2012 9:20 pm Post subject: |
|
|
Try some simplified netfilter rules temporarily; one MARK for WAN1/2 each. It might be also important to do some SNAT, otherwise some packets will have the wrong source IP (i.e. the IP of the other interface). You should make some tcpdumps with your tests. _________________ Death by snoo-snoo! |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Wed Oct 10, 2012 10:44 pm Post subject: |
|
|
papahuhn wrote: | Try some simplified netfilter rules temporarily; one MARK for WAN1/2 each. It might be also important to do some SNAT, otherwise some packets will have the wrong source IP (i.e. the IP of the other interface). You should make some tcpdumps with your tests. |
I have SNAT setup, forgot to mention it, here it is:
Code: | /sbin/iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -m mark --mark 0 -j SNAT --to-source $WAN2
/sbin/iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -m mark --mark 1 -j SNAT --to-source $WAN1
/sbin/iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -m mark --mark 2 -j SNAT --to-source $WAN2 |
Do you recommend to add more lines for -s 127.0.0.1, or just ! -d 192.168.0.0 to include also localhost? _________________ Practice Makes Perfect |
|
Back to top |
|
|
b0nafide Apprentice
Joined: 17 Feb 2008 Posts: 171 Location: ~/
|
Posted: Wed Oct 10, 2012 10:58 pm Post subject: |
|
|
On the router, try
Code: | route add 190.2.0.0/24 gw 190.2.0.foo
route add 66.220.158.0/24 gw 66.220.158.bar
|
I don't see what you're trying to achieve with all the connmark'ing, but perhaps I'm missing something. |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Wed Oct 10, 2012 11:03 pm Post subject: |
|
|
b0nafide wrote: | On the router, try
Code: | route add 190.2.0.0/24 gw 190.2.0.foo
route add 66.220.158.0/24 gw 66.220.158.bar
|
I don't see what you're trying to achieve with all the connmark'ing, but perhaps I'm missing something. |
I think that the gw there should be my WAN1 or WAN2 addressess, but it would be having to do everything twice -.-
I have 2 ISP connections, but since they don't have the same WAN IP i can't bond them together so I'm marking some networks to go thru one or the other ISP, it works fine for LAN users, except for the router itself, which allways uses the default route instead of the iptables marked one.
Although with the last reply about the SNAT I think he nailed it, since it's only "SNATting" the packets comming from 192.168.0.0 and excluding the ones from localhost, perhaps the packets are properly marked but the source is not being changed...
Code: | /sbin/iptables -t nat -A POSTROUTING -s 127.0.0.1 -d 0.0.0.0 -m mark --mark 2 -j SNAT --to-source $WAN2
/sbin/iptables -t nat -A POSTROUTING ! -s 192.168.0.0/16 -d 0.0.0.0 -m mark --mark 0 -j SNAT --to-source $WAN2
/sbin/iptables -t nat -A POSTROUTING ! -s 0.0.0.0 -m mark --mark 0 -j SNAT --to-source $WAN2 |
Which one of these would you recommend me to try? _________________ Practice Makes Perfect |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Wed Oct 10, 2012 11:12 pm Post subject: |
|
|
Here's the tcpdump for visiting icanhazip.com from my router...
Code: | 20:10:33.970170 IP WAN1.10320 > 198.61.150.28.80: SWE 3879243526:3879243526(0) win 5840 <mss 1460>
20:10:34.179445 IP 198.61.150.28.80 > WAN1.10320: SE 147161378:147161378(0) ack 3879243527 win 14600 <mss 1460>
20:10:34.179461 IP WAN1.10320 > 198.61.150.28.80: . ack 1 win 5840
20:10:34.179537 IP WAN1.10320 > 198.61.150.28.80: P 1:602(601) ack 1 win 5840
20:10:34.368430 IP 198.61.150.28.80 > WAN1.10320: P 1:327(326) ack 602 win 15626
20:10:34.368447 IP WAN1.10320 > 198.61.150.28.80: . ack 327 win 6432
20:10:34.397445 IP 198.61.150.28.80 > WAN1.10320: . ack 602 win 15626
20:10:34.397455 IP WAN1.10320 > 198.61.150.28.80: . ack 327 win 6432
20:10:36.015355 IP WAN1.10320 > 198.61.150.28.80: F 602:602(0) ack 327 win 6432
20:10:36.230296 IP 198.61.150.28.80 > WAN1.10320: F 327:327(0) ack 603 win 15626
20:10:36.230311 IP WAN1.10320 > 198.61.150.28.80: . ack 328 win 6432 |
And here's the one from visiting it thru my LAN computer:
Code: | 20:11:21.253430 IP WAN2.53645 > 198.61.150.28.80: S 4136494562:4136494562(0) win 8192 <mss 1460,nop,nop,sackOK>
20:11:21.253540 IP WAN2.53646 > 198.61.150.28.80: S 3314477280:3314477280(0) win 8192 <mss 1460,nop,nop,sackOK>
20:11:21.306034 IP WAN2.53647 > 198.61.150.28.80: S 1287074544:1287074544(0) win 8192 <mss 1460,nop,nop,sackOK>
20:11:21.428756 IP 198.61.150.28.80 > WAN2.53645: S 1151037725:1151037725(0) ack 4136494563 win 14600 <mss 1460,nop,nop,sackOK>
20:11:21.430475 IP WAN2.53645 > 198.61.150.28.80: . ack 1 win 17520
20:11:21.431106 IP 198.61.150.28.80 > WAN2.53646: S 3590466767:3590466767(0) ack 3314477281 win 14600 <mss 1460,nop,nop,sackOK>
20:11:21.432504 IP WAN2.53645 > 198.61.150.28.80: P 1:383(382) ack 1 win 17520
20:11:21.438041 IP WAN2.53646 > 198.61.150.28.80: . ack 1 win 17520
20:11:21.484339 IP 198.61.150.28.80 > WAN2.53647: S 1385340538:1385340538(0) ack 1287074545 win 14600 <mss 1460,nop,nop,sackOK>
20:11:21.486884 IP WAN2.53647 > 198.61.150.28.80: . ack 1 win 17520
20:11:21.612309 IP 198.61.150.28.80 > WAN2.53645: . ack 383 win 15544
20:11:21.613295 IP 198.61.150.28.80 > WAN2.53645: P 1:330(329) ack 383 win 15544
20:11:21.809934 IP WAN2.53645 > 198.61.150.28.80: P 383:716(333) ack 330 win 17191
20:11:21.988099 IP 198.61.150.28.80 > WAN2.53645: P 330:661(331) ack 716 win 16616
20:11:22.189800 IP WAN2.53645 > 198.61.150.28.80: . ack 661 win 16860 |
As you can see one of these has WAN1 (the first one) and the second has WAN2 (my LAN), the first one is wrong, it should go out thru the IP of WAN2.
PS: I've changed the gateway IPs manually to WAN1 and WAN2, tcpdump shows the actual IP, but I don't want to show them publicly. _________________ Practice Makes Perfect |
|
Back to top |
|
|
b0nafide Apprentice
Joined: 17 Feb 2008 Posts: 171 Location: ~/
|
Posted: Wed Oct 10, 2012 11:30 pm Post subject: |
|
|
kuteninja wrote: | I think that the gw there should be my WAN1 or WAN2 addressess |
Hmmm whatever your gateways are could be determined with something like Code: | grep -i "gateway=" `find /var/lib/ -name "dhcpcd-eth*"` |
or, something better like
Code: | route add 190.2.0.0/24 gw `sed "s/"\'"/ /g" /var/lib/dhcpcd-eth0.info | awk '/^GATEWAY=/ { print $2 }'` |
If you knew that particular gateway was always going to be eth0
Last edited by b0nafide on Wed Oct 10, 2012 11:44 pm; edited 2 times in total |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Wed Oct 10, 2012 11:34 pm Post subject: |
|
|
I have static dedicated IPs on both of the ISPs, you're not getting the case, I'd like to avoid routing without passing thru iptables mangle -> nat first (from localhost to wan)
On your example you made me edit the route, I don't want to add a route for each one that's on my mangle iptables, I want localhost to go thru the mangle table. _________________ Practice Makes Perfect |
|
Back to top |
|
|
b0nafide Apprentice
Joined: 17 Feb 2008 Posts: 171 Location: ~/
|
Posted: Wed Oct 10, 2012 11:42 pm Post subject: |
|
|
Ok, I understand now. My setup does what I need it to at the moment without mangling packets. |
|
Back to top |
|
|
kuteninja n00b
Joined: 07 Sep 2009 Posts: 58
|
Posted: Wed Oct 10, 2012 11:44 pm Post subject: |
|
|
I got it fixed, as I noticed the packets were being marked as mark2, but the SNAT wasn't catching them, so I did this:
- If the packet is trying to go out "from" WAN1 (using the default gateway) but with a mark2, then it should be SNAT to WAN2
- If the packet is trying to go out "from" WAN2 (if the default gateway gets changed) but with a mark1, then it should be SNAT to WAN1
That did the trick. _________________ Practice Makes Perfect |
|
Back to top |
|
|
|