Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] linux bridge not forwarding traffic outside
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
viacheslavg
n00b
n00b


Joined: 23 Sep 2016
Posts: 8

PostPosted: Tue Aug 31, 2021 2:25 pm    Post subject: [SOLVED] linux bridge not forwarding traffic outside Reply with quote

I'm trying to setup network namespace which later will be used by dedicated user. The idea is that this user will play with network configurations (e.g. setup VPN) but this changes should not affect network on host (i.e. "default" network namespace).
What I did so far:
1. Setup network namespace along with veth pair:
Code:

ip netns add vnet0
ip link add veth0 type veth peer name eth0 netns vnet0

2. Configure interfaces and routes in namespace:
Code:

ip netns exec vnet0 ip link set lo up
ip netns exec vnet0 ip addr add 192.168.8.2/24 dev eth0
ip netns exec vnet0 ip link set eth0 up
ip netns exec vnet0 ip route add default via 192.168.8.1 dev eth0

3. Create bridge in the host and add veth there:
Code:

ip link add name br0 type bridge
ip addr add 192.168.8.1/24 brd 192.168.8.255 dev br0
ip link set veth0 master br0
ip link set br0 up

As result I have following on host:
Code:

# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 2e:9e:3f:99:fc:c9 brd ff:ff:ff:ff:ff:ff
3: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
4: enp2s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether e4:7f:b2:17:26:49 brd ff:ff:ff:ff:ff:ff
5: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 80:19:34:c0:e6:8a brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.120/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp3s0
       valid_lft 41981sec preferred_lft 41981sec
    inet6 fd63:61e4:95ff::f97/128 scope global dynamic noprefixroute
       valid_lft 85184sec preferred_lft 85184sec
    inet6 fd63:61e4:95ff:0:1cdd:1f29:e279:ca55/64 scope global dynamic noprefixroute
       valid_lft 5984sec preferred_lft 584sec
    inet6 fe80::5957:17e5:842f:3cd8/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
8: veth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether ae:de:a7:6d:0d:dd brd ff:ff:ff:ff:ff:ff link-netns vnet0
    inet6 fe80::d04:c570:6e2d:fdc3/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
9: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ae:de:a7:6d:0d:dd brd ff:ff:ff:ff:ff:ff
    inet 192.168.8.1/24 brd 192.168.8.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::acde:a7ff:fe6d:ddd/64 scope link
       valid_lft forever preferred_lft forever

# ip route
default via 192.168.1.1 dev wlp3s0 proto dhcp metric 600
192.168.1.0/24 dev wlp3s0 proto kernel scope link src 192.168.1.120 metric 600
192.168.8.0/24 dev br0 proto kernel scope link src 192.168.8.1

and following inside network namespace:
Code:

# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
3: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 3a:d2:8c:56:cc:6b brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.8.2/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::38d2:8cff:fe56:cc6b/64 scope link
       valid_lft forever preferred_lft forever

# ip route
default via 192.168.8.1 dev eth0
192.168.8.0/24 dev eth0 proto kernel scope link src 192.168.8.2


My problem is that I'm not able to reach any IP from within this namespace outside of host.
e.g.:
Code:

# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2082ms

What I'm doing wrong? Could it be that packets are somehow filtered in bridge? I have no special rules in iptables:
Code:

# iptables -t filter -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         

# ebtables -t filter -L
Bridge table: filter

Bridge chain: INPUT, entries: 0, policy: ACCEPT

Bridge chain: FORWARD, entries: 0, policy: ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

# ebtables -t nat -L
Bridge table: nat

Bridge chain: PREROUTING, entries: 0, policy: ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 0, policy: ACCEPT

bridge also looks good:
Code:

# brctl showstp br0
br0
 bridge id      8000.aedea76d0ddd
 designated root   8000.aedea76d0ddd
 root port         0         path cost         0
 max age        20.00         bridge max age        20.00
 hello time         2.00         bridge hello time      2.00
 forward delay        15.00         bridge forward delay     15.00
 ageing time       300.00
 hello timer         0.00         tcn timer         0.00
 topology change timer      0.00         gc timer       201.75
 flags         


veth0 (1)
 port id      8001         state           forwarding
 designated root   8000.aedea76d0ddd   path cost         2
 designated bridge   8000.aedea76d0ddd   message age timer      0.00
 designated port   8001         forward delay timer      0.00
 designated cost      0         hold timer         0.00
 flags         


As an additional observation: if i start docker, which will create docker0 bridge and bunch of rules in iptables and add my veth0 interface to docker0 bridge then everything works fine. I have no glue why docker0 bridge works fine but mine not.
Any idea?


Last edited by viacheslavg on Fri Sep 03, 2021 11:19 am; edited 1 time in total
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3095

PostPosted: Tue Aug 31, 2021 3:01 pm    Post subject: Reply with quote

Depending on your network, you should either:
* enable ip forwarding (via sysctl) and add MASQUERADE rule to POSTROUTING in nat table - most common.
* bridge your veth with a physical devices (add your wifi to the bridge and move wifi's IP there too).
* (rare - if you don't know this is your use case, it is not) just enable ip forwarding.
Back to top
View user's profile Send private message
viacheslavg
n00b
n00b


Joined: 23 Sep 2016
Posts: 8

PostPosted: Wed Sep 01, 2021 3:46 pm    Post subject: Reply with quote

Thanks for helping.
As I mentioned earlier everything works out of the box if i move my veth to the docker0 bridge (created by docker). I don't need to put wifi to the bridge enable IP forwarding etc.
I replicated all the iptable rules (which docker creates for docker0) for my bridge (including MASQUERADE you mention). so now I have:
Code:
# iptables -t nat -v -L
Chain PREROUTING (policy ACCEPT 23 packets, 3192 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   26  3130 DOCKER     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 5 packets, 1432 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 424 packets, 28043 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  any    any     anywhere            !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 424 packets, 28043 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    3   252 MASQUERADE  all  --  any    !docker0  192.168.64.0/24      anywhere           
    0     0 MASQUERADE  all  --  any    !br-68c3e5c395af  192.168.70.0/24      anywhere           
    0     0 MASQUERADE  all  --  any    !br0    192.168.8.0/24       anywhere           

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  docker0 any     anywhere             anywhere           
    0     0 RETURN     all  --  br-68c3e5c395af any     anywhere             anywhere           
    0     0 RETURN     all  --  br0    any     anywhere             anywhere           

# iptables -t filter -v -L
Chain INPUT (policy ACCEPT 1152 packets, 459K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 4 packets, 336 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   80  6664 DOCKER-USER  all  --  any    any     anywhere             anywhere           
   80  6664 DOCKER-ISOLATION-STAGE-1  all  --  any    any     anywhere             anywhere           
   13  1092 ACCEPT     all  --  any    docker0  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    9   756 DOCKER     all  --  any    docker0  anywhere             anywhere           
   13  1092 ACCEPT     all  --  docker0 !docker0  anywhere             anywhere           
    0     0 ACCEPT     all  --  docker0 docker0  anywhere             anywhere           
    0     0 ACCEPT     all  --  any    br-68c3e5c395af  anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  any    br-68c3e5c395af  anywhere             anywhere           
    0     0 ACCEPT     all  --  br-68c3e5c395af !br-68c3e5c395af  anywhere             anywhere           
    0     0 ACCEPT     all  --  br-68c3e5c395af br-68c3e5c395af  anywhere             anywhere           
    0     0 ACCEPT     all  --  any    br0     anywhere             anywhere           
    0     0 DOCKER     all  --  any    br0     anywhere             anywhere           
    0     0 ACCEPT     all  --  br0    !br0    anywhere             anywhere           
    0     0 ACCEPT     all  --  br0    br0     anywhere             anywhere           

Chain OUTPUT (policy ACCEPT 1230 packets, 160K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (3 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br0    !br0    anywhere             anywhere           
   13  1092 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  anywhere             anywhere           
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  br-68c3e5c395af !br-68c3e5c395af  anywhere             anywhere           
   80  6664 RETURN     all  --  any    any     anywhere             anywhere           

Chain DOCKER-ISOLATION-STAGE-2 (3 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  any    br0     anywhere             anywhere           
    0     0 DROP       all  --  any    docker0  anywhere             anywhere           
    0     0 DROP       all  --  any    br-68c3e5c395af  anywhere             anywhere           
   13  1092 RETURN     all  --  any    any     anywhere             anywhere           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   80  6664 RETURN     all  --  any    any     anywhere             anywhere           

still no go. I suspect docker does yet more "magic" with it's docker0 bridge. But what else could it be? no idea....maybe some extra netfilter table I'm not aware of?
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3095

PostPosted: Wed Sep 01, 2021 4:23 pm    Post subject: Reply with quote

If you showed us the result of iptables-save instead of iptables -vL, it would be easier to follow.
It provides an unambiguous dump (one that can be imported back). It will also include all tables without any need to manually specify them. And I suppose many people here are more used to its format (yes, including me).

Docker is a really messy piece of software. It alters your firewall rules in weird ways, it can disable your firewall, drop some traffic you actually wanted, bang your milk or drink your cat.
If you want to run it alongside anything else on the same host, you should probably isolate it with another network namespace. It can be done and is not very hard once you know how to use namespaces (AFAIK it breaks swarm mode, but pretty much any other docker manager will work, should you need it) so you're almost there: just create another ns, connect it to the host with a pair of veth and run docker daemon inside. And either bridge or route traffic as needed.
This basically requires you to write your own init scripts; but it's still not very hard to do.
Back to top
View user's profile Send private message
viacheslavg
n00b
n00b


Joined: 23 Sep 2016
Posts: 8

PostPosted: Fri Sep 03, 2021 11:18 am    Post subject: Reply with quote

Indeed iptables-save output is much better and it helped me to sort out the problem. Docker was also involved but I can't say it does silly things.
Here is explanation:
I had /etc/sysctl.d/local.conf with
Code:
net.ipv4.ip_forward=1

so, I thought ip forwarding is enabled, but as it appears in /etc/sysctl.conf I had:
Code:
net.ipv4.ip_forward=0

and as result forwarding was disabled. Which is strange, my assumption was that custom configs in sysctl.d/ should override the one from /etc/sysctl.conf
However when docker starts it enables this option and to preserve existing behavior for the rest of system it changes FORWARD policy from ACCEPT to DROP (plus of course adds rules for it's own bridge to make it working).
Thus when I start my system w/o docker enabled forwarding was not working because of net.ipv4.ip_forward=0 setting, but when I start my system with docker forwarding was disabled for my bridge in iptables by docker.
Summarizing, the solution is following:
Possible two options:
1. If you are OK with having net.ipv4.ip_forward=1 system wide then only following rule required in iptables:
Code:
iptables -t nat -A POSTROUTING -s 192.168.8.0/24 -j MASQUERADE

2. If you need to have net.ipv4.ip_forward=0 and docker is installed then two additional rules can fix the problem:
Code:

iptables -A FORWARD -o br0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i br0 ! -o br0 -j ACCEPT


Thanks a lot for helping!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum