Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[FIXED] How to route the router's traffic
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Tue Oct 09, 2012 7:58 pm    Post subject: [FIXED] How to route the router's traffic Reply with quote

Hard to get title... well this is the case:

I have a linux router (gentoo) with 2 gateways WAN1 and WAN2.
The same machine have a LAN where all my machines are connected with UPNP, DHCP and such.

As a router it works perfectly, I'm using this method:

LAN user wants to go to 1.2.3.4
iptables mangle marks everything going to 1.2.3.4 as mark1
ip route knows it should go thru WAN1 (using table1)
all further packets to 1.2.3.4 go thru WAN1

LAN user wants to go to 33.33.33.33
iptables mangle marks everything going to 33.33.33.33 as mark2
ip route knows it should go thru WAN2 (using table2)
all further packets to 33.33.33.33 go thru WAN2

But... if i visit 33.33.33.33 from the router itself (ping, wget, etc), iptables doesn't do anything and it uses the default gateway which is WAN1 instead of going out thru WAN2 like it should have.

How can I force localhost to decide where to go using iptables mangle instead of using the default gateway directly?
_________________
Practice Makes Perfect


Last edited by kuteninja on Wed Oct 10, 2012 11:47 pm; edited 1 time in total
Back to top
View user's profile Send private message
papahuhn
l33t
l33t


Joined: 06 Sep 2004
Posts: 600

PostPosted: Tue Oct 09, 2012 8:57 pm    Post subject: Reply with quote

You need to mangle in OUTPUT for that.
_________________
Death by snoo-snoo!
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Tue Oct 09, 2012 8:58 pm    Post subject: Reply with quote

papahuhn wrote:
You need to mangle in OUTPUT for that.

Oh, so the local traffic from the router goes directly thru mangle->output and the LAN traffic goes thru mangle->prerouting or I should use only output?
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
papahuhn
l33t
l33t


Joined: 06 Sep 2004
Posts: 600

PostPosted: Tue Oct 09, 2012 8:59 pm    Post subject: Reply with quote

Use both.
_________________
Death by snoo-snoo!
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Tue Oct 09, 2012 9:05 pm    Post subject: Reply with quote

papahuhn wrote:
Use both.

Nope, still not working.

I've setup the same connmark rules on output and I'm still going out thru the default gateway instead of the iptables rule.
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
papahuhn
l33t
l33t


Joined: 06 Sep 2004
Posts: 600

PostPosted: Tue Oct 09, 2012 9:15 pm    Post subject: Reply with quote

Do you only use CONNMARK without packet MARK? The latter is actually relevant for policy routing.
_________________
Death by snoo-snoo!
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Tue Oct 09, 2012 9:45 pm    Post subject: Reply with quote

Let's make it easier, here's my iptables mangle setup:

CONNMARK1 = WAN1
CONNMARK2 = WAN2

Code:
-A PREROUTING -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A PREROUTING -d 192.168.0.0/16 -j RETURN
-A PREROUTING -d 10.0.0.0/24 -j RETURN
-A PREROUTING -d 190.2.0.0/24 -m state --state NEW -j CONNMARK1
-A PREROUTING -d 66.220.158.0/24 -m state --state NEW -j CONNMARK2
-A PREROUTING -m mark ! --mark 0x1 -m state --state NEW -j CONNMARK2
-A PREROUTING -m mark --mark 0x0 -m state --state NEW -j CONNMARK2

-A OUTPUT -m state --state RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A OUTPUT -d 192.168.0.0/16 -j RETURN
-A OUTPUT -d 10.0.0.0/24 -j RETURN
-A OUTPUT -d 190.2.0.0/24 -m state --state NEW -j CONNMARK1
-A OUTPUT -d 66.220.158.0/24 -m state --state NEW -j CONNMARK2
-A OUTPUT -m mark ! --mark 0x1 -m state --state NEW -j CONNMARK2
-A OUTPUT -m mark --mark 0x0 -m state --state NEW -j CONNMARK2

-A CONNMARK1 -j MARK --set-xmark 0x1/0xffffffff
-A CONNMARK1 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff

-A CONNMARK2 -j MARK --set-xmark 0x2/0xffffffff
-A CONNMARK2 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff


Let's say I visit 190.2.0.1 from my computer on the LAN, I go thru WAN1, then if I visit 66.220.158.1 I go thru WAN2.

I do the same on the router and I allways go thru WAN1 since it's the default gateway on route table main when I should use WAN1 or WAN2 depending on what iptables says.
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Wed Oct 10, 2012 8:29 pm    Post subject: Reply with quote

Sad bump :(

I couldn't find what's wrong.

I tried to remove the default gateway on the main table; the LAN kept having internet access but my router can't ping or browse by itself.
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
papahuhn
l33t
l33t


Joined: 06 Sep 2004
Posts: 600

PostPosted: Wed Oct 10, 2012 9:20 pm    Post subject: Reply with quote

Try some simplified netfilter rules temporarily; one MARK for WAN1/2 each. It might be also important to do some SNAT, otherwise some packets will have the wrong source IP (i.e. the IP of the other interface). You should make some tcpdumps with your tests.
_________________
Death by snoo-snoo!
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Wed Oct 10, 2012 10:44 pm    Post subject: Reply with quote

papahuhn wrote:
Try some simplified netfilter rules temporarily; one MARK for WAN1/2 each. It might be also important to do some SNAT, otherwise some packets will have the wrong source IP (i.e. the IP of the other interface). You should make some tcpdumps with your tests.


I have SNAT setup, forgot to mention it, here it is:

Code:
/sbin/iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -m mark --mark 0 -j SNAT --to-source $WAN2
/sbin/iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -m mark --mark 1 -j SNAT --to-source $WAN1
/sbin/iptables -t nat -A POSTROUTING -s 192.168.0.0/16 -m mark --mark 2 -j SNAT --to-source $WAN2


Do you recommend to add more lines for -s 127.0.0.1, or just ! -d 192.168.0.0 to include also localhost?
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
b0nafide
Apprentice
Apprentice


Joined: 17 Feb 2008
Posts: 153
Location: ~/

PostPosted: Wed Oct 10, 2012 10:58 pm    Post subject: Reply with quote

On the router, try
Code:
route add 190.2.0.0/24 gw 190.2.0.foo
route add 66.220.158.0/24 gw 66.220.158.bar

I don't see what you're trying to achieve with all the connmark'ing, but perhaps I'm missing something.
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Wed Oct 10, 2012 11:03 pm    Post subject: Reply with quote

b0nafide wrote:
On the router, try
Code:
route add 190.2.0.0/24 gw 190.2.0.foo
route add 66.220.158.0/24 gw 66.220.158.bar

I don't see what you're trying to achieve with all the connmark'ing, but perhaps I'm missing something.


I think that the gw there should be my WAN1 or WAN2 addressess, but it would be having to do everything twice -.-

I have 2 ISP connections, but since they don't have the same WAN IP i can't bond them together so I'm marking some networks to go thru one or the other ISP, it works fine for LAN users, except for the router itself, which allways uses the default route instead of the iptables marked one.

Although with the last reply about the SNAT I think he nailed it, since it's only "SNATting" the packets comming from 192.168.0.0 and excluding the ones from localhost, perhaps the packets are properly marked but the source is not being changed...

Code:
/sbin/iptables -t nat -A POSTROUTING -s 127.0.0.1 -d 0.0.0.0 -m mark --mark 2 -j SNAT --to-source $WAN2
/sbin/iptables -t nat -A POSTROUTING ! -s 192.168.0.0/16 -d 0.0.0.0 -m mark --mark 0 -j SNAT --to-source $WAN2
/sbin/iptables -t nat -A POSTROUTING ! -s 0.0.0.0 -m mark --mark 0 -j SNAT --to-source $WAN2


Which one of these would you recommend me to try?
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Wed Oct 10, 2012 11:12 pm    Post subject: Reply with quote

Here's the tcpdump for visiting icanhazip.com from my router...

Code:
20:10:33.970170 IP WAN1.10320 > 198.61.150.28.80: SWE 3879243526:3879243526(0) win 5840 <mss 1460>
20:10:34.179445 IP 198.61.150.28.80 > WAN1.10320: SE 147161378:147161378(0) ack 3879243527 win 14600 <mss 1460>
20:10:34.179461 IP WAN1.10320 > 198.61.150.28.80: . ack 1 win 5840
20:10:34.179537 IP WAN1.10320 > 198.61.150.28.80: P 1:602(601) ack 1 win 5840
20:10:34.368430 IP 198.61.150.28.80 > WAN1.10320: P 1:327(326) ack 602 win 15626
20:10:34.368447 IP WAN1.10320 > 198.61.150.28.80: . ack 327 win 6432
20:10:34.397445 IP 198.61.150.28.80 > WAN1.10320: . ack 602 win 15626
20:10:34.397455 IP WAN1.10320 > 198.61.150.28.80: . ack 327 win 6432
20:10:36.015355 IP WAN1.10320 > 198.61.150.28.80: F 602:602(0) ack 327 win 6432
20:10:36.230296 IP 198.61.150.28.80 > WAN1.10320: F 327:327(0) ack 603 win 15626
20:10:36.230311 IP WAN1.10320 > 198.61.150.28.80: . ack 328 win 6432


And here's the one from visiting it thru my LAN computer:

Code:
20:11:21.253430 IP WAN2.53645 > 198.61.150.28.80: S 4136494562:4136494562(0) win 8192 <mss 1460,nop,nop,sackOK>
20:11:21.253540 IP WAN2.53646 > 198.61.150.28.80: S 3314477280:3314477280(0) win 8192 <mss 1460,nop,nop,sackOK>
20:11:21.306034 IP WAN2.53647 > 198.61.150.28.80: S 1287074544:1287074544(0) win 8192 <mss 1460,nop,nop,sackOK>
20:11:21.428756 IP 198.61.150.28.80 > WAN2.53645: S 1151037725:1151037725(0) ack 4136494563 win 14600 <mss 1460,nop,nop,sackOK>
20:11:21.430475 IP WAN2.53645 > 198.61.150.28.80: . ack 1 win 17520
20:11:21.431106 IP 198.61.150.28.80 > WAN2.53646: S 3590466767:3590466767(0) ack 3314477281 win 14600 <mss 1460,nop,nop,sackOK>
20:11:21.432504 IP WAN2.53645 > 198.61.150.28.80: P 1:383(382) ack 1 win 17520
20:11:21.438041 IP WAN2.53646 > 198.61.150.28.80: . ack 1 win 17520
20:11:21.484339 IP 198.61.150.28.80 > WAN2.53647: S 1385340538:1385340538(0) ack 1287074545 win 14600 <mss 1460,nop,nop,sackOK>
20:11:21.486884 IP WAN2.53647 > 198.61.150.28.80: . ack 1 win 17520
20:11:21.612309 IP 198.61.150.28.80 > WAN2.53645: . ack 383 win 15544
20:11:21.613295 IP 198.61.150.28.80 > WAN2.53645: P 1:330(329) ack 383 win 15544
20:11:21.809934 IP WAN2.53645 > 198.61.150.28.80: P 383:716(333) ack 330 win 17191
20:11:21.988099 IP 198.61.150.28.80 > WAN2.53645: P 330:661(331) ack 716 win 16616
20:11:22.189800 IP WAN2.53645 > 198.61.150.28.80: . ack 661 win 16860


As you can see one of these has WAN1 (the first one) and the second has WAN2 (my LAN), the first one is wrong, it should go out thru the IP of WAN2.

PS: I've changed the gateway IPs manually to WAN1 and WAN2, tcpdump shows the actual IP, but I don't want to show them publicly.
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
b0nafide
Apprentice
Apprentice


Joined: 17 Feb 2008
Posts: 153
Location: ~/

PostPosted: Wed Oct 10, 2012 11:30 pm    Post subject: Reply with quote

kuteninja wrote:
I think that the gw there should be my WAN1 or WAN2 addressess


Hmmm whatever your gateways are could be determined with something like
Code:
grep -i "gateway=" `find /var/lib/ -name "dhcpcd-eth*"`


or, something better like

Code:
route add 190.2.0.0/24 gw `sed "s/"\'"/ /g" /var/lib/dhcpcd-eth0.info | awk '/^GATEWAY=/ { print $2 }'`


If you knew that particular gateway was always going to be eth0


Last edited by b0nafide on Wed Oct 10, 2012 11:44 pm; edited 2 times in total
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Wed Oct 10, 2012 11:34 pm    Post subject: Reply with quote

I have static dedicated IPs on both of the ISPs, you're not getting the case, I'd like to avoid routing without passing thru iptables mangle -> nat first (from localhost to wan)

On your example you made me edit the route, I don't want to add a route for each one that's on my mangle iptables, I want localhost to go thru the mangle table.
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
b0nafide
Apprentice
Apprentice


Joined: 17 Feb 2008
Posts: 153
Location: ~/

PostPosted: Wed Oct 10, 2012 11:42 pm    Post subject: Reply with quote

Ok, I understand now. My setup does what I need it to at the moment without mangling packets.
Back to top
View user's profile Send private message
kuteninja
n00b
n00b


Joined: 07 Sep 2009
Posts: 58

PostPosted: Wed Oct 10, 2012 11:44 pm    Post subject: Reply with quote

I got it fixed, as I noticed the packets were being marked as mark2, but the SNAT wasn't catching them, so I did this:

- If the packet is trying to go out "from" WAN1 (using the default gateway) but with a mark2, then it should be SNAT to WAN2
- If the packet is trying to go out "from" WAN2 (if the default gateway gets changed) but with a mark1, then it should be SNAT to WAN1

That did the trick.
_________________
Practice Makes Perfect
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum