Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[SOLVED] KVM and network bridge
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
AgBr
Apprentice
Apprentice


Joined: 06 Nov 2010
Posts: 186

PostPosted: Fri Jun 01, 2012 3:49 pm    Post subject: [SOLVED] KVM and network bridge Reply with quote

I need help with the setup of network configuration of my Host system.
Goal is to set up a virtual hub so that the guests are on the same subnet as the host and are visible to other hosts on the net as if they were physical machines.
I tried to follow this instruction. To my opinion the instruction is somewhat ambiguous as there are two concepts (virtual hub and NAT/masquerading) mixed up. May be I didn't understand it all. So I need a little help here.

Up till now I have one FreeBSD guest (elrond) running, to which I have access from the host (dragon) via ssh. From the guest I can see the host but can not see any other machine on the net (featuring celeborn as an example below) as well as I can not get into the guest from any other machine except from the virtualization host.
My configuration so far looks like this:

Code:

dragon ~ # ifconfig eth0
eth0      Protokoll:Ethernet  Hardware Adresse 00:19:99:b9:8d:2d 
          inet6 Adresse: fe80::219:99ff:feb9:8d2d/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:45111 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4828 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:1000
          RX bytes:6556353 (6.2 MiB)  TX bytes:472331 (461.2 KiB)
          Interrupt:18 Speicher:cd420000-cd440000

dragon ~ # ifconfig br0
br0       Protokoll:Ethernet  Hardware Adresse 00:00:00:00:00:02 
          inet Adresse:172.16.1.4  Bcast:172.16.31.255  Maske:255.255.224.0
          inet6 Adresse: fe80::200:ff:fe00:2/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23760 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3716 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:0
          RX bytes:3601507 (3.4 MiB)  TX bytes:368598 (359.9 KiB)

dragon ~ # ifconfig tap0
tap0      Protokoll:Ethernet  Hardware Adresse 00:00:00:00:00:02 
          UP BROADCAST PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:1 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:500
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

dragon ~ # ifconfig tap1
tap1      Protokoll:Ethernet  Hardware Adresse 00:00:00:00:00:03 
          inet6 Adresse: fe80::200:ff:fe00:3/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:444 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18557 errors:0 dropped:1 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:500
          RX bytes:38115 (37.2 KiB)  TX bytes:2090768 (1.9 MiB)

dragon ~ # brctl show br0
bridge name     bridge id               STP enabled     interfaces
br0             8000.000000000002       no              eth0
                                                        tap0
                                                        tap1
       
dragon ~ # sysctl net.ipv4.conf.eth0
net.ipv4.conf.eth0.forwarding = 0
net.ipv4.conf.eth0.mc_forwarding = 0
net.ipv4.conf.eth0.accept_redirects = 1
net.ipv4.conf.eth0.secure_redirects = 1
net.ipv4.conf.eth0.shared_media = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.eth0.send_redirects = 1
net.ipv4.conf.eth0.accept_source_route = 1
net.ipv4.conf.eth0.accept_local = 0
net.ipv4.conf.eth0.src_valid_mark = 0
net.ipv4.conf.eth0.proxy_arp = 0
net.ipv4.conf.eth0.medium_id = 0
net.ipv4.conf.eth0.bootp_relay = 0
net.ipv4.conf.eth0.log_martians = 0
net.ipv4.conf.eth0.tag = 0
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.arp_announce = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.eth0.arp_accept = 0
net.ipv4.conf.eth0.arp_notify = 0
net.ipv4.conf.eth0.proxy_arp_pvlan = 0
net.ipv4.conf.eth0.disable_xfrm = 0
net.ipv4.conf.eth0.disable_policy = 0
net.ipv4.conf.eth0.force_igmp_version = 0
net.ipv4.conf.eth0.promote_secondaries = 0

dragon ~ # sysctl net.ipv4.conf.br0
net.ipv4.conf.br0.forwarding = 0
net.ipv4.conf.br0.mc_forwarding = 0
net.ipv4.conf.br0.accept_redirects = 1
net.ipv4.conf.br0.secure_redirects = 1
net.ipv4.conf.br0.shared_media = 1
net.ipv4.conf.br0.rp_filter = 1
net.ipv4.conf.br0.send_redirects = 1
net.ipv4.conf.br0.accept_source_route = 1
net.ipv4.conf.br0.accept_local = 0
net.ipv4.conf.br0.src_valid_mark = 0
net.ipv4.conf.br0.proxy_arp = 1
net.ipv4.conf.br0.medium_id = 0
net.ipv4.conf.br0.bootp_relay = 0
net.ipv4.conf.br0.log_martians = 0
net.ipv4.conf.br0.tag = 0
net.ipv4.conf.br0.arp_filter = 0
net.ipv4.conf.br0.arp_announce = 1
net.ipv4.conf.br0.arp_ignore = 0
net.ipv4.conf.br0.arp_accept = 0
net.ipv4.conf.br0.arp_notify = 0
net.ipv4.conf.br0.proxy_arp_pvlan = 0
net.ipv4.conf.br0.disable_xfrm = 0
net.ipv4.conf.br0.disable_policy = 0
net.ipv4.conf.br0.force_igmp_version = 0
net.ipv4.conf.br0.promote_secondaries = 0

dragon ~ # sysctl net.ipv4.conf.tap0
net.ipv4.conf.tap0.forwarding = 0
net.ipv4.conf.tap0.mc_forwarding = 0
net.ipv4.conf.tap0.accept_redirects = 1
net.ipv4.conf.tap0.secure_redirects = 1
net.ipv4.conf.tap0.shared_media = 1
net.ipv4.conf.tap0.rp_filter = 1
net.ipv4.conf.tap0.send_redirects = 1
net.ipv4.conf.tap0.accept_source_route = 1
net.ipv4.conf.tap0.accept_local = 0
net.ipv4.conf.tap0.src_valid_mark = 0
net.ipv4.conf.tap0.proxy_arp = 0
net.ipv4.conf.tap0.medium_id = 0
net.ipv4.conf.tap0.bootp_relay = 0
net.ipv4.conf.tap0.log_martians = 0
net.ipv4.conf.tap0.tag = 0
net.ipv4.conf.tap0.arp_filter = 0
net.ipv4.conf.tap0.arp_announce = 0
net.ipv4.conf.tap0.arp_ignore = 0
net.ipv4.conf.tap0.arp_accept = 0
net.ipv4.conf.tap0.arp_notify = 0
net.ipv4.conf.tap0.proxy_arp_pvlan = 0
net.ipv4.conf.tap0.disable_xfrm = 0
net.ipv4.conf.tap0.disable_policy = 0
net.ipv4.conf.tap0.force_igmp_version = 0
net.ipv4.conf.tap0.promote_secondaries = 0

dragon ~ # sysctl net.ipv4.conf.tap1
net.ipv4.conf.tap1.forwarding = 0
net.ipv4.conf.tap1.mc_forwarding = 0
net.ipv4.conf.tap1.accept_redirects = 1
net.ipv4.conf.tap1.secure_redirects = 1
net.ipv4.conf.tap1.shared_media = 1
net.ipv4.conf.tap1.rp_filter = 1
net.ipv4.conf.tap1.send_redirects = 1
net.ipv4.conf.tap1.accept_source_route = 1
net.ipv4.conf.tap1.accept_local = 0
net.ipv4.conf.tap1.src_valid_mark = 0
net.ipv4.conf.tap1.proxy_arp = 0
net.ipv4.conf.tap1.medium_id = 0
net.ipv4.conf.tap1.bootp_relay = 0
net.ipv4.conf.tap1.log_martians = 0
net.ipv4.conf.tap1.tag = 0
net.ipv4.conf.tap1.arp_filter = 0
net.ipv4.conf.tap1.arp_announce = 0
net.ipv4.conf.tap1.arp_ignore = 0
net.ipv4.conf.tap1.arp_accept = 0
net.ipv4.conf.tap1.arp_notify = 0
net.ipv4.conf.tap1.proxy_arp_pvlan = 0
net.ipv4.conf.tap1.disable_xfrm = 0
net.ipv4.conf.tap1.disable_policy = 0
net.ipv4.conf.tap1.force_igmp_version = 0
net.ipv4.conf.tap1.promote_secondaries = 0

dragon ~ # sysctl net.bridge
net.bridge.bridge-nf-call-arptables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-filter-vlan-tagged = 0
net.bridge.bridge-nf-filter-pppoe-tagged = 0

dragon ~ # lsmod
Module                  Size  Used by
iptable_filter           984  0
ip_tables               7681  1 iptable_filter
x_tables                9464  2 iptable_filter,ip_tables
kvm_intel             107912  3
kvm                   243019  1 kvm_intel
tun                     9802  5
bridge                 52516  0
ipv6                  205677  17 bridge
stp                     1051  1 bridge
llc                     2538  2 bridge,stp
r8169                  34883  0
mptsas                 27327  1
8250_pnp                3391  0
mptscsih               14369  1 mptsas
i7core_edac            12552  0
i2c_i801                6174  0
edac_core              24556  3 i7core_edac
mptbase                44384  2 mptsas,mptscsih
serio_raw               2913  0
mii                     2823  1 r8169
ac                      2373  0
e1000e                107805  0
8250                   15537  1 8250_pnp
serial_core            12029  1 8250

dragon ~ # netstat -rn
Kernel IP Routentabelle
Ziel            Router          Genmask         Flags   MSS Fenster irtt Iface
0.0.0.0         172.16.20.5     0.0.0.0         UG        0 0          0 br0
127.0.0.0       -               255.0.0.0       !         - -          - -
172.16.0.0      0.0.0.0         255.255.224.0   U         0 0          0 br0


As you can see from the arp-Tables, arp seems to work

Code:

dragon ~ # arp -a
celeborn (172.16.1.2) auf 00:19:99:7d:fa:78 [ether] auf br0
elrond (172.16.1.7) auf 00:00:00:00:00:03 [ether] auf br0

celeborn# arp -a|grep 172.16.1.7
? (172.16.1.7) at 00:00:00:00:00:03 on bge0 expires in 1146 seconds [ethernet]

celeborn# arp -a | grep 172.16.1.4
dragon (172.16.1.4) at 00:00:00:00:00:02 on bge0 expires in 1087 seconds [ethernet]

elrond# arp -a
? (172.16.1.4) at 00:00:00:00:00:02 on re0 expires in 907 seconds [ethernet]
? (172.16.1.7) at 00:00:00:00:00:03 on re0 permanent [ethernet]
? (172.16.1.2) at 00:19:99:7d:fa:78 on re0 expires in 1115 seconds [ethernet]
 



The VM is attached to tap1. tap0 isn't in use at this point. Interestingly br0 has the same MAC as tap0.
The guest is started with
Code:

qemu-kvm -hda /usr/local/util/freebsd.img -hdb /dev/sdb \
                -net nic,macaddr=00:00:00:00:00:03 -net tap,ifname=tap1,script=no,downscript=no \
                -cdrom /dev/sr0 -curses -k de 2


I have tried fiddeling with proxy_arp and forwarding to no avail. As far as I understand at least forwarding should not matter here, as arp shows the tap Interfaces outside the host-Machine (on celeborn).

Iptables isn't involved either:
Code:

dragon ~ # iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination


So what am I missing here?


Last edited by AgBr on Wed Jun 06, 2012 8:46 am; edited 1 time in total
Back to top
View user's profile Send private message
Hu
Watchman
Watchman


Joined: 06 Mar 2007
Posts: 8829

PostPosted: Fri Jun 01, 2012 10:39 pm    Post subject: Reply with quote

Please post a command that fails to connect in the way you want, and the full output generated by that command. Also post the output of /sbin/ip route or the equivalent for both endpoints of the attempted connection.
Back to top
View user's profile Send private message
AgBr
Apprentice
Apprentice


Joined: 06 Nov 2010
Posts: 186

PostPosted: Fri Jun 01, 2012 11:25 pm    Post subject: Reply with quote

Hu wrote:
Please post a command that fails to connect in the way you want, and the full output generated by that command. Also post the output of /sbin/ip route or the equivalent for both endpoints of the attempted connection.

Code:

user@gandalf ~ $ ssh root@172.16.1.7
ssh: connect to host 172.16.1.7 port 22: Connection timed out
user@gandalf ~ $ ping 172.16.1.7
PING 172.16.1.7 (172.16.1.7) 56(84) bytes of data.
Ping doesn't return anything ^C after quite some while.

user@gandalf ~ $ netstat -rn
Kernel IP Routentabelle
Ziel            Router          Genmask         Flags   MSS Fenster irtt Iface
172.16.0.0      0.0.0.0         255.255.224.0   U         0 0          0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
0.0.0.0         172.16.20.5     0.0.0.0         UG        0 0          0 eth0

user@gandalf ~ $ /sbin/arp -a
dragon (172.16.1.4) auf 00:00:00:00:00:02 [ether] auf eth0
celeborn (172.16.1.2) auf 00:19:99:7d:fa:78 [ether] auf eth0
? (172.16.1.7) auf 00:00:00:00:00:03 [ether] auf eth0
ulli@gandalf ~ $

Logged into the guest (elrond) via the host (dragon) and ping to gandalf (172.16.20.9)
Code:

elrond# ping 172.16.20.9
PING 172.16.20.9 (172.16.20.9): 56 data bytes
^C
--- 172.16.20.9 ping statistics ---
60 packets transmitted, 0 packets received, 100.0% packet loss
elrond# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            172.16.20.5        UGS         0        0    re0
127.0.0.1          link#3             UH          0       25    lo0
172.16.0.0/19      link#1             U           0     1385    re0
172.16.1.7         link#1             UHS         0        0    lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               ::1                           UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%re0/64                     link#1                        U           re0
fe80::200:ff:fe00:3%re0           link#1                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff01::%re0/32                     fe80::200:ff:fe00:3%re0       U           re0
ff01::%lo0/32                     ::1                           U           lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%re0/32                     fe80::200:ff:fe00:3%re0       U           re0
ff02::%lo0/32                     ::1                           U           lo0
elrond# arp -a
? (172.16.20.9) at 00:30:05:40:68:aa on re0 expires in 1042 seconds [ethernet]
? (172.16.1.4) at 00:00:00:00:00:02 on re0 expires in 1061 seconds [ethernet]
? (172.16.1.7) at 00:00:00:00:00:03 on re0 permanent [ethernet]
? (172.16.1.2) at 00:19:99:7d:fa:78 on re0 expires in 1141 seconds [ethernet]


Arp-Tables presented again for your convenience. As you can see, arp does work. If arp works, the theory is, that layer 2 must work. Otherwise the machines would not have any means to know the mac-addresses for the target-interfaces of the respective IP-addresses. If layer 2 works, on the local net IP-traffic should work too as there is no router involved. I can log into the guest (172.16.1.7/19) from the host (172.16.1.4) but not from any other machine on the same subnet (172.16.20.9/19 or 172.16.1.2/19) for instance)
Back to top
View user's profile Send private message
AngelKnight
Tux's lil' helper
Tux's lil' helper


Joined: 14 Jan 2003
Posts: 118

PostPosted: Mon Jun 04, 2012 2:04 pm    Post subject: Reply with quote

You didn't mention whether or not you turned forwarding on on the host, with

Code:
sysctl net.ipv4.ip_forward=1
Back to top
View user's profile Send private message
AgBr
Apprentice
Apprentice


Joined: 06 Nov 2010
Posts: 186

PostPosted: Mon Jun 04, 2012 3:06 pm    Post subject: Reply with quote

AngelKnight wrote:
You didn't mention whether or not you turned forwarding on on the host, with

Code:
sysctl net.ipv4.ip_forward=1



I have tried this too despite that it should not be necessary. It did not make a difference.
Back to top
View user's profile Send private message
AngelKnight
Tux's lil' helper
Tux's lil' helper


Joined: 14 Jan 2003
Posts: 118

PostPosted: Tue Jun 05, 2012 6:08 pm    Post subject: Reply with quote

Argh, yes you're correct: there's no L3 forwarding here. Sorry, I thought I'd read more carefully.

If I read this correctly, you set the host's tap1 to be a forced MAC (0000.0000.0003) -and- set the inside guest to use the same forced MAC.

If you did, don't: let the host's tap interface float and use qemu-kvm commandline to force the MAC on the adapter inside. I've already tested this; when the host TAP has the same MAC as the guest's simulated ethernet, the host TAP will tend to eat the frames, not pass them through. Don't worry about collisions too much: the autoassigned address has most-sig-byte's bit 1 set (marking it local-admin).

I vaguely remember being bit by this but evidently misremembered why earlier, sorry about that.
Back to top
View user's profile Send private message
AgBr
Apprentice
Apprentice


Joined: 06 Nov 2010
Posts: 186

PostPosted: Tue Jun 05, 2012 6:51 pm    Post subject: Reply with quote

AngelKnight wrote:
Argh, yes you're correct: there's no L3 forwarding here. Sorry, I thought I'd read more carefully.

If I read this correctly, you set the host's tap1 to be a forced MAC (0000.0000.0003) -and- set the inside guest to use the same forced MAC.

If you did, don't: let the host's tap interface float and use qemu-kvm commandline to force the MAC on the adapter inside. I've already tested this; when the host TAP has the same MAC as the guest's simulated ethernet, the host TAP will tend to eat the frames, not pass them through. Don't worry about collisions too much: the autoassigned address has most-sig-byte's bit 1 set (marking it local-admin).

I vaguely remember being bit by this but evidently misremembered why earlier, sorry about that.


I was following the above mentioned instructions here but I will try that. Meanwhile I have found out by contemplating some tcpdump output, that all broadcast frames go in through the tap interface and all direct frames from the guest and all broadcasts go out through the tap but not through the bridge-Interface. So all frames having specific MAC-Addresses coming from the tap are blocked by the bridge interface.
The box in question is remote. I hope fiddeling with the tap interface will not kill the bridge. Otherwise I will have to wait until tomorrow.
Back to top
View user's profile Send private message
AgBr
Apprentice
Apprentice


Joined: 06 Nov 2010
Posts: 186

PostPosted: Wed Jun 06, 2012 8:45 am    Post subject: Reply with quote

AngelKnight wrote:

... let the host's tap interface float and use qemu-kvm commandline to force the MAC on the adapter inside.


The problem is solved this way. Thank you. The wiki seems to be wrong in this respect.
Back to top
View user's profile Send private message
AngelKnight
Tux's lil' helper
Tux's lil' helper


Joined: 14 Jan 2003
Posts: 118

PostPosted: Wed Jun 06, 2012 11:01 pm    Post subject: Reply with quote

This wiki specifically says to make the host machine the defaultrouter in the guest, not the actual network's bridge. This is probably why.

Trying to use the real network segment's defaultrouter will not work for this configuration as you have discovered.

This wiki needs quite a bit of fixing around networking actually. Ugh. I'm too drunk to fix this now. Anyone? :D
Back to top
View user's profile Send private message
AgBr
Apprentice
Apprentice


Joined: 06 Nov 2010
Posts: 186

PostPosted: Thu Jun 07, 2012 6:20 am    Post subject: Reply with quote

AngelKnight wrote:
This wiki specifically says to make the host machine the defaultrouter in the guest, not the actual network's bridge. This is probably why.

Trying to use the real network segment's defaultrouter will not work for this configuration as you have discovered.

This wiki needs quite a bit of fixing around networking actually. Ugh. I'm too drunk to fix this now. Anyone? :D


I am shy to do it as my knowledge about these things is too limited as you know. But I have made a comment about my experiences in the talk section already with reference to this thread.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum