| View previous topic :: View next topic |
| Author |
Message |
Marbug
n00b
Joined: 08 Jun 2012
Posts: 5
|
 Posted: Fri Jun 08, 2012 3:07 pm Post subject: Bridging br0 over eth1 gives slow pings and packet loss |
 |
|
I'm using linux-vserver in combination with kvm, so I'm using the latest stable linux-vserver kernel (3.4.0-vs2.3.3.4).
I followed the kvm wiki @ http://en.gentoo-wiki.com/wiki/KVM guid.
Every time br0 is started there are a lot of packet loss and connections can't be made to the outside world.
it's like there isn't a dns anymore. But I can still connect to the server for a while. And after a time it's even impossible to connect to it.
When I make a ping on the server to google, it takes a while to get a reaction, and often some or all packets are lost.
as first I noticed I also needed to enable "Bridged IP/ARP packets filtering" in the kernel:
| Code: |
Bridged IP/ARP packets filtering
Bridged IP/ARP packets filtering
Network packet filtering framework (Netfilter)
[*] Bridged IP/ARP packets filtering
|
802.1d Ethernet Bridging in Network options wasn't enough because at boot time it gave me errors on the following in /etc/sysctl.conf about invalid keys
| Code: |
net.bridge.bridge-nf-call-arptables=0
net.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-ip6tables=0
|
my network configuration is:
eth0 for linux-vserver & host
eth1 for br0
my /etc/conf.d/net
| Code: |
config_eth0=( "192.168.2.120/24" )
routes_eth0=( "default via 192.168.2.1" )
bridge_br0="eth1 tap0 tap1 tap2 tap3"
brctl_br0="setfd 0 sethello 0 stp off"
rc_need_br0="net.tap0 net.tap1 net.tap2 net.tap3"
config_br0="192.168.2.110/24"
routes_br0="default via 192.168.2.1"
config_tap0="null"
tuntap_tap0="tap"
tunctl_tap0="-u kvmuser"
mac_tap0="00:00:00:00:00:00"
config_tap1="null"
tuntap_tap1="tap"
tunctl_tap1="-u kvmuser"
mac_tap1="00:00:00:00:00:01"
config_tap2="null"
tuntap_tap2="tap"
tunctl_tap2="-u kvmuser"
mac_tap2="00:00:00:00:00:02"
config_tap3="null"
tuntap_tap3="tap"
tunctl_tap3="-u kvmuser"
mac_tap3="00:00:00:00:00:03"
config_eth1="null"
|
is it also normal that my router shows 00:00:00:00:00:01 for the mac adress of 192.168.2.110? |
|
| Back to top |
|
 |
Rexilion
Veteran
Joined: 17 Mar 2009
Posts: 1044
|
| Posted: Fri Jun 08, 2012 3:27 pm Post subject: |
|
|
I suggest you try a more simpler approach (any specific reason why you need bridging?)
Simply try this:
First remove all the previous configurations, and reboot your computer. You need a clean start. Now execute the following commands:
The 'ip' command is from the 'iproute2' package.
First, we setup a simple tap interface to attach your VM network to your hosts network:
| Quote: | ip tuntap add dev tap0 mode tap user marbug group marbug
ip addr add 10.28.67.1/24 dev tap0
ip link set tap0 up |
Replace 'marbug' with the username that the KVM instance will be running as.
Second, we prepare iptables to handle this traffic:
| Quote: | iptables -t nat -A POSTROUTING -m addrtype ! --src-type LOCAL ! --dst-type LOCAL -j MASQUERADE --random
iptables -P FORWARD ACCEPT |
The addrtype statements ('! --src-type LOCAL' and '! --dst-type LOCAL') is an optimization. It will only MASQUERADE traffic that really needs it therefore reducing the load on your CPU.
The '--random' switch mitigates some security risks.
The second line asks traffic that is not directed at your computer, but does pass through it, to be allowed.
Third, we ask the linux kernel to reset it's IP parameters suitable for traffic routing:
| Quote: | sysctl -q net.ipv4.conf.all.forwarding=1
sysctl -q net.ipv4.conf.default.forwarding=1 |
All should be set, try to start the kvm with at least something like this:
| Quote: | | qemu-kvm -net nic,macaddr=00:1d:92:ab:3f:78 -net tap,ifname=tap0,script=no,downscript=no |
Make sure that you assign a valid ip to the VM while it's running! Valid IP's are in the range of 10.28.67.1/24, which means that anything from 10.28.67.2 till 10.28.67.255 will do. Yes, it could also be 10.28.67.1, but we assigned that to our tap0 device. An example for ip configuration in the VM:
| Quote: | IP: 10.28.67.66
Default route 10.28.67.1
Netmask 255.255.255.0
Primary DNS: 208.67.222.222
Secondary DNS: 208.67.220.220 |
Please let us know how it works out!
_________________
fs/super.c: "Self-destruct in 5 seconds. Have a nice day...\n" |
|
| Back to top |
|
|
Marbug
n00b
Joined: 08 Jun 2012
Posts: 5
|
| Posted: Fri Jun 08, 2012 11:59 pm Post subject: |
|
|
Thanks for the nice and fast reply,
I'm using bridging because I would like to give a virtual machine it's own ip.
I used vde (virtual switch) before but since I'm using libvirtd with virt-manager I can't specify vde and I can only specify a bridged name for the source device.
vde was more simple to do but on the other hand less clean.
after searching a while on the iptables line I found out that I needed to enable the addrtype in my kernel
so I did and I continue following you're steps, but when I wanted to launch kvm the problem started.
I'm using virt-manager so I had set tap0 as ethernet device, but when I want to boot up my virtual machine, it gave me the following error:
| Code: |
Error starting domain: Unable to add bridge tap0 port vnet0: Operation not supported
|
I can remember I had the same problem when I was searching myself, and I remember I needed to use br0 instead
so isn't the br0 device forgotten in this?
and oh I forgot to mention that the solution I said was also done on this computer (my desktop).
It has a tuxonice kernel (3.0.17-r1) with only 1 Ethernet card (eth0), and it works fine.
The /etc/conf.d/net on my desktop:
| Code: |
bridge_br0="eth0 tap0 tap1 tap2 tap3 tap4 tap5"
brctl_br0="setfd 0 sethello 0 stp off"
rc_need_br0="net.tap0 net.tap1 net.tap2 net.tap3 net.tap4 net.tap5"
config_br0="192.168.2.15/24"
routes_br0="default via 192.168.2.1"
dns_servers_br0="192.168.2.1"
config_tap0="null"
tuntap_tap0="tap"
tunctl_tap0="-u gentoo"
mac_tap0="00:00:00:00:00:00"
config_tap1="null"
tuntap_tap1="tap"
tunctl_tap1="-u gentoo"
mac_tap1="00:00:00:00:00:01"
config_tap2="null"
tuntap_tap2="tap"
tunctl_tap2="-u gentoo"
mac_tap2="00:00:00:00:00:02"
config_tap3="null"
tuntap_tap3="tap"
tunctl_tap3="-u gentoo"
mac_tap3="00:00:00:00:00:03"
config_tap4="null"
tuntap_tap4="tap"
tunctl_tap4="-u gentoo"
mac_tap4="00:00:00:00:00:04"
config_tap5="null"
tuntap_tap5="tap"
tunctl_tap5="-u gentoo"
mac_tap5="00:00:00:00:00:05"
config_eth0="null"
|
|
|
| Back to top |
|
|
Rexilion
Veteran
Joined: 17 Mar 2009
Posts: 1044
|
| Posted: Sat Jun 09, 2012 7:30 am Post subject: |
|
|
Good thing you mention that it works on your laptop. That really helps. Some questions and a suggestion.
| Quote: | | Error starting domain: Unable to add bridge tap0 port vnet0: Operation not supported |
That is probably my mistake, your kvm-qemu command should be:
| Quote: | | qemu-kvm -net nic -net tap,ifname=tap0,script=no,downscript=no |
The problem that it no longer makes sense to define a mac. Ow btw, did you know that you *can* access these computers from other computers like any normal network? It only requires that the other computers have a route. Assuming your ip to your regular network is 192.168.2.120, they can do:
| Quote: | | ip route 10.28.67.0/24 dev eth0 via 192.168.2.120 |
You can add these custom routes to NetworkManager. But of course, this is not required with bridging. However, bridging is a quite intrusive operation and not all network cards support it (properly).
| Quote: | config_eth0=( "192.168.2.120/24" )
routes_eth0=( "default via 192.168.2.1" )
bridge_br0="eth1 tap0 tap1 tap2 tap3"
brctl_br0="setfd 0 sethello 0 stp off"
rc_need_br0="net.tap0 net.tap1 net.tap2 net.tap3"
config_br0="192.168.2.110/24"
routes_br0="default via 192.168.2.1" |
This denotes the difference between your laptop and your desktop. On your laptop, the first 2 lines are missing. I sense a collission in your configuration. This could be your rootcause of your problem. This could also explain the eventual 'dying' of the connection, this could be related to the ARP cache timeout.
| Quote: | net.bridge.bridge-nf-call-arptables=0
net.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-ip6tables=0 |
Do you get the same errors about these rules on your laptop?
And as a final remark, someone else is running into issue's with this as well. They mention some solutions of which one worked:
https://forums.gentoo.org/viewtopic-t-925466.html
_________________
fs/super.c: "Self-destruct in 5 seconds. Have a nice day...\n" |
|
| Back to top |
|
|
Marbug
n00b
Joined: 08 Jun 2012
Posts: 5
|
| Posted: Sat Jun 09, 2012 9:07 am Post subject: |
|
|
It is a desktop not a laptop, but thats a detail 
the problem is on my server which has 2 network cards
I tried not to use a mac adress in virt-manager but it won't accept it then.
So I couldn't try it with the sollution you gave.
Anyway I must say I learned a bit of iproutes and the ip command as I never had used one of those.
I always configured through config files and I can say I still like that as when you reboot then everything is still there.
Ofcorse you're solution lets everything still works when you reboot when you made a mistake 
| Quote: |
ip route 10.28.67.0/24 dev eth0 via 192.168.2.120
|
and oh, I didn't knew that, so I always gave 192.168.2.1/24 as the iprange instead of the 10.28.67.1/24
It could be that the support of bridging could be the problem.
On this computer the following card is used as eth0:
| Code: |
05:00.0 Ethernet controller: Atheros Communications Inc. AR8151 v2.0 Gigabit Ethernet (rev c0)
|
while on my server eth0 is the first card, while eth1 is the 2nd line:
| Code: |
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 09)
05:02.0 Ethernet controller: 3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)
|
I don't know how old the 3Com card is but I needed a 2nd card when I used the vde (virtual switch) otherwise I couldn't access my host from the kvm machines. (this is/was a kernel limitation)
I must also note that my first solution was to use 1 Ethernet card, but something went wrong, I don't know anymore what. Let me reproduce it
well ... when I had it configured with 1 ethernet card, I gave my linux-vservers the br0 device, and even if I give eth1 to them and used br0 on eth0 I had no ethernet access on the server, nor could I connect to it, nor could I connect to the outside world. ifconfig gave some clean results, only the ip of eth1 was the last ip specified to a linux-vserver.
| Quote: |
This denotes the difference between your laptop and your desktop. On your laptop, the first 2 lines are missing. I sense a collission in your configuration. This could be your rootcause of your problem. This could also explain the eventual 'dying' of the connection, this could be related to the ARP cache timeout.
|
The reason why the 2 lines are missing is because I have only 1 ethernet card in my laptop (actually my desktop, my server got 2 ethernet cards)
I thought the problem could be that there was a problem, that it maybe send through the 2 cards or that the buffer got filled or so.
As it wasn't possible to connect to the server anymore, so I don't think that would be a timeout of the ARP cache.
| Quote: |
Do you get the same errors about these rules on your laptop?
|
At first I didn't had the same errors, but after a few reboots it gave me the same errors out of nothing. Or maybe I changed something, but nothing related to ip or something. it was another thing I was solving in the tuxonice kernel for another purpose. But I enabled the same option in the kernel as I did in the kernel om my server and it was solved again:
| Code: |
Bridged IP/ARP packets filtering
Bridged IP/ARP packets filtering
Network packet filtering framework (Netfilter)
[*] Bridged IP/ARP packets filtering
|
I can remember that I had found the same thread with google.
I need to do something else right now, I'll reread the thread in less than an houre.
Maybe I had missed something or so. |
|
| Back to top |
|
|
Marbug
n00b
Joined: 08 Jun 2012
Posts: 5
|
| Posted: Sat Jun 09, 2012 11:34 am Post subject: |
|
|
I was reading about the mac address in that thread, and that was when I noticed something, the tap devices on my server and this computer where identical the same. So they had the same mac addresses. I changed the mac addresses on my server, and now it works. Knowing that I searched for that stupid detail for more than 10 hours 
Quote of the day:
| Quote: |
Stupid details are only found after hours of searching and editing >_<
|
/etc/conf.d/net on my server:
| Code: |
# This blank configuration will automatically use DHCP for any net.*
# scripts in /etc/init.d. To create a more complete configuration,
# please review /etc/conf.d/net.example and save your configuration
# in /etc/conf.d/net (this file :]!).
config_eth0="192.168.2.120/24"
routes_eth0="default via 192.168.2.1"
dns_servers_eth0="192.168.2.1"
#config_eth1=( "192.168.2.110/24" )
#routes_eth1=( "default via 192.168.2.1" )
config_eth1="null"
bridge_br0="eth1 tap0 tap1 tap2 tap3"
brctl_br0="setfd 0 sethello 0 stp off"
rc_need_br0="net.tap0 net.tap1 net.tap2 net.tap3"
config_br0="192.168.2.110/24"
routes_br0="default via 192.168.2.1"
dns_servers_br0="192.168.2.1"
config_tap0="null"
tuntap_tap0="tap"
tunctl_tap0="-u kvmuser"
mac_tap0="00:00:00:00:01:00"
config_tap1="null"
tuntap_tap1="tap"
tunctl_tap1="-u kvmuser"
mac_tap1="00:00:00:00:01:01"
config_tap2="null"
tuntap_tap2="tap"
tunctl_tap2="-u kvmuser"
mac_tap2="00:00:00:00:01:02"
config_tap3="null"
tuntap_tap3="tap"
tunctl_tap3="-u kvmuser"
mac_tap3="00:00:00:00:01:03"
|
/etc/conf.d/net on my desktop:
| Code: |
# This blank configuration will automatically use DHCP for any net.*
# scripts in /etc/init.d. To create a more complete configuration,
# please review /usr/share/doc/openrc*/net.example* and save your configuration
# in /etc/conf.d/net (this file :]!).
#config_eth0="192.168.2.15 netmask 255.255.255.0 brd 192.168.2.255"
#routes_eth0="default via 192.168.2.1"
bridge_br0="eth0 tap0 tap1 tap2 tap3 tap4 tap5"
brctl_br0="setfd 0 sethello 0 stp off"
rc_need_br0="net.tap0 net.tap1 net.tap2 net.tap3 net.tap4 net.tap5"
config_br0="192.168.2.15/24"
routes_br0="default via 192.168.2.1"
dns_servers_br0="192.168.2.1"
config_tap0="null"
tuntap_tap0="tap"
tunctl_tap0="-u gentoo"
mac_tap0="00:00:00:00:00:00"
config_tap1="null"
tuntap_tap1="tap"
tunctl_tap1="-u gentoo"
mac_tap1="00:00:00:00:00:01"
config_tap2="null"
tuntap_tap2="tap"
tunctl_tap2="-u gentoo"
mac_tap2="00:00:00:00:00:02"
config_tap3="null"
tuntap_tap3="tap"
tunctl_tap3="-u gentoo"
mac_tap3="00:00:00:00:00:03"
config_eth0="null"
|
Now everything works fine! |
|
| Back to top |
|
|
Rexilion
Veteran
Joined: 17 Mar 2009
Posts: 1044
|
| Posted: Sat Jun 09, 2012 11:43 am Post subject: |
|
|
Glad it worked out. You will never forget this one 
_________________
fs/super.c: "Self-destruct in 5 seconds. Have a nice day...\n" |
|
| Back to top |
|
|
Marbug
n00b
Joined: 08 Jun 2012
Posts: 5
|
| Posted: Sat Jun 09, 2012 12:38 pm Post subject: |
|
|
I must say that I still needed to change some little things as I noticed my dns on the server was gone when a virtual computer in kvm was started. I found out when I removed any configuration to the br0 everything worked again, also in the virtual computer ethernet worked fine:
| Code: |
# This blank configuration will automatically use DHCP for any net.*
# scripts in /etc/init.d. To create a more complete configuration,
# please review /etc/conf.d/net.example and save your configuration
# in /etc/conf.d/net (this file :]!).
config_eth0="192.168.2.120/24"
routes_eth0="default via 192.168.2.1"
dns_servers_eth0="192.168.2.1"
#config_eth1=( "192.168.2.110/24" )
#routes_eth1=( "default via 192.168.2.1" )
bridge_br0="eth1 tap0 tap1 tap2 tap3"
brctl_br0="setfd 0 sethello 0 stp off"
rc_need_br0="net.tap0 net.tap1 net.tap2 net.tap3"
config_br0="null"
config_eth1="null"
config_tap0="null"
tuntap_tap0="tap"
tunctl_tap0="-u kvmuser"
mac_tap0="00:00:00:00:01:00"
config_tap1="null"
tuntap_tap1="tap"
tunctl_tap1="-u kvmuser"
mac_tap1="00:00:00:00:01:01"
config_tap2="null"
tuntap_tap2="tap"
tunctl_tap2="-u kvmuser"
mac_tap2="00:00:00:00:01:02"
config_tap3="null"
tuntap_tap3="tap"
tunctl_tap3="-u kvmuser"
mac_tap3="00:00:00:00:01:03"
|
|
|
| Back to top |
|
|
Rexilion
Veteran
Joined: 17 Mar 2009
Posts: 1044
|
| Posted: Sat Jun 09, 2012 12:57 pm Post subject: |
|
|
That is correct, the bridge itself doesn't need a valid IP address. Everything goes automagically over the interface which has the default route (unless another route is specified that covers the destination IP, like the route via example I gave you).
The interface with the default route now services connections with two MAC-addresses. One from itself, and another from tap0. That is why it's so important they were unique since that is the attribute which is used to split and merge the data coming from the devices behind the bridge.
_________________
fs/super.c: "Self-destruct in 5 seconds. Have a nice day...\n" |
|
| Back to top |
|
|
AngelKnight
Tux's lil' helper
Joined: 14 Jan 2003
Posts: 127
|
| Posted: Sat Jun 09, 2012 5:16 pm Post subject: |
|
|
| Argh, dammit. That's two too many people who've tripped on that because of bad advice. That wiki link on gentoo-wiki has been patched up now. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
