Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Confused about KVM Network Configuration
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
dustfinger
Guru
Guru


Joined: 15 Aug 2004
Posts: 449

PostPosted: Mon Oct 05, 2015 7:56 am    Post subject: Confused about KVM Network Configuration Reply with quote

Hi,
I am completely confused about networking a qemu kvm. I will mention now that I have not yet created the vm. I am trying to sort out the networking so that when I run the command to create the vm I can pass the network details.
Goal: to create a kvm that has its own static IP address.
Current Setup
The host has a single physical network adapter and the interface eth0 configured with a static IP address and a vlan eth0.0 configured with a different static IP address.

/etc/conf.d/net
Code:

config_eth00="<static-ip-1> netmask 255.255.255.0 brd <static-ip-1-fragment>.255"
routes_enp3s0f0="default via <static-ip-1-fragment>.1"

vlans_eth0="2"

eth0_vlan2_name="vlan2l"
config_vlan2="<static-ip-2> netmask 255.255.255.0 brd <static-ip-2-fragment>.255"


I have the second IP address for the kvm. I created vlan2 with the intention that the kvm would use that interface. I have since read that I need to use a bridge. I have read the wiki on networking with qemu as well as as the wiki on network bridging and am thoroughly confused.I think that part of my confusion stems from the fact that the wiki does not explain what it intends to achieve with the particular configuration that they outline. Simply put, I am confused because:
1. Their example does not use a vlan.
2. They set config_eth0=null whereas I am starting off with eth0 set to a static ip and my vlan eth0.2 set to a static ip.
3. They configure the bridge interface br0 to ether dhcp or a default local IP.

My Questions:
1. Why do I really need a bridge?
2. Should I be setting my vlan in the same way that they wiki sets eth0: config_vlan2=null and config_br0="<static-ip-2> netmask 255.255.255.0 brd <static-ip-2-fragment>.255" then bridge_br0="vlan2 tap0"
3. How can I go about configuring /etc/conf.d/net so that I can reach my goals as described above?

Sincerely,
dustfinger

-- EDIT: --
#Here is my command for creating the vm.
#Note that I am creating it on a zvol
#So far this does not work :-(

Code:

VNIC=vlan2
HDD=tank/vm/prod-web-1 #My zvol
CD=~/vm/livedvd-amd64-multilib-20140826.iso
mac=`ip link show vlan2 | awk '/ether/ {print $2}'`

qemu-system-x86_64 -enable-kvm \
        -boot cd \
        -global ide-drive.physical_block_size=4096 \
        -drive file=$HDD,if=virtio,index=0,cach=none,format=raw \
        -drive file=$CD,media=cdrom,if=ide,index=2 \
        -netdev tap,id=t0,ifname=$VNIC,script=no,downscript=no
        -device e1000,netdev=t0
        -machine pc,accel=kvm,iommu=on \
        -cpu host \
        -smp 4 \
        -m 8G,slots=2,maxmem=16G \
        -name prod-web-1 \
        -usb \
        -nographic

----
_________________
Unanswered Post Initiative:
https://forums.gentoo.org/viewtopic.php?t=119906
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43753
Location: 56N 3W

PostPosted: Mon Oct 05, 2015 7:48 pm    Post subject: Reply with quote

dustfinger,

A bridge is the software equivelent of a router - every packet sent to the bridge goes to every device connected to the bridge.

On the host, you set
Code:
config_eth0="null"
so that you can donate the interface to a bridge.
The bridge gets an IP address in the normal way and works on the host as would eth0.

You use the tun/tap driver to export connections from the bridge to your VMs.
Inside your VMs they have a network interface that works in the normal way.
They can use dhcp to get an address from your dhcp server or you can set up the net file with a static address.

Everything connected to the bridge will be in the same subnet as the bare metal host.

The bare metal setup .. all static.
Code:
config_eth0="null"
config_eth1="null"
config_eth2="null"
config_eth3="null"
config_eth4="null"

# My public subnet
config_br0="xx.yy.zz.ww/29"

# the DMZ
bridge_br1="eth2"
config_br1="192.168.10.254/24"

# wireless
bridge_br2="eth3"
config_br2="192.168.54.254/24"

# protected wired
bridge_br3="eth4"
config_br3="192.168.100.254/24"


The bridges are all passed to a VM to do firewalling, where they appear as eth0 ... eth3.

The VM is started as
Code:
/usr/bin/qemu-system-x86_64 -name Router -S -machine pc-0.14,accel=kvm,usb=off -m 1024 -realtime mlock=off \
-smp 2,sockets=2,cores=1,threads=1 -uuid 19860cc8-b232-1cad-c562-15b18883886a -no-user-config -nodefaults\
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/Router.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control\
-rtc base=utc -no-shutdown -boot menu=off,strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
-drive file=/dev/vm/router,if=none,id=drive-virtio-disk0,format=raw \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio\disk0,bootindex=1 \
-netdev tap,fd=20,id=hostnet0,vhost=on,vhostfd=21 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:2b:f1:3e,bus=pci.0,addr=0x6 \
-netdev tap,fd=22,id=hostnet1,vhost=on,vhostfd=23 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:b7:f5:01,bus=pci.0,addr=0x5 \
-netdev tap,fd=24,id=hostnet2,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet2,id=net2,mac=52:54:00:ad:25:ea,bus=pci.0,addr=0x7 \
-netdev tap,fd=26,id=hostnet3,vhost=on,vhostfd=27 -device virtio-net-pci,netdev=hostnet3,id=net3,mac=52:54:00:ab:b1:41,bus=pci.0,addr=0x8 \
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on

I didn't write that command line, virt-manager did.
The use of the tap device to connect tho the host bridges can clearly be seen.

Why don't you want to use a bridge?
From the inside, the VM behaves in all respects as another system, even though its a program executing on the host.

Inside the KVM, I have
Code:
config_eth0="192.168.10.253/24 brd 192.168.10.255"
config_eth1="192.168.100.253/24 brd 192.168.100.255"
config_eth2="null"
config_eth3="192.168.54.253/24 brd 192.168.54.255"


config_eth2="null" is because the router is a PPPoE end point, so no IP address is needed on the interface that provides the PPPoE link.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
dustfinger
Guru
Guru


Joined: 15 Aug 2004
Posts: 449

PostPosted: Tue Oct 06, 2015 3:58 am    Post subject: bridges for one physical ethernet with two ip addresses? Reply with quote

Hi NeddySeagoon,

Thank you so much for your time and advice. In your example you show how you bridge several physical interfaces: eth0, eth1, eth3 etc. I only have one network card and so I have only one physical interface: eth0. I do have 2 static ip addresses though. I would like the host to have one static ip address and the vm to have the other static ip address. In the example below I attempt to follow your pattern, but I create a vlan off of the physical interface for the second IP address. I then try to create two bridges, one for eth0 and one for my vlan (web1). Unfortunately this did not work. The result was that tap0, eth0 and web1 all appeared to start, but did not have the static IP address assigned to them. Trying to start /etc/init.d/net.br0 would just complain that br0 required services web1 and tap0. Trying to start /etc/init.d/net.br1 would just complain that net.br1 failed to start.

Code:

#So that I can donate the interface to a bridge
config_eth0=null
#so that I can have a vlan for my second IP address
vlans_eth0="2"
eth0_vlan2_name="web1"
#so that I can donate the interface to a bridge
config_web1=null

#Configure TUN/TAP interface
tuntap_tap0="tap"

# tap0 defined empty
config_tap0=null

# Configure network bridge for the physical interface eth0
config_br0="198.xx.xx.225 netmask 255.255.255.0 brd 198.xx.xx.255"
routes_br0="default via 198.xx.xx.1"
mac_br0="aa:bb:cc:dd:ee:ff"
bridge_br0="eth0" # add all interfaces to bridge
rc_net_br0_need="eth0" # we need run eth0 before create bridge!

# Configure network bridge for the vlan interface
config_br1="192.yy.yy.241 netmask 255.255.255.0 brd 192.yy.yy.255"
mac_br1="aa:bb:cc:dd:ee:ff" #
bridge_br1="web1 tap0" # add all interfaces to bridge, usually use one TUN/TAP interface for one Vritual Machine. In this example we have one vm
rc_net_br1_need="web1 tap0" # we need run web1 and tap0 before create bridge!


How can I correctly make use of both my static IP addresses with only one physical ethernet?

Sincerely,

dustfinger.


-- EDIT --
Hi,
I decided to take a step back and configure a much simpler senario. So I decided to only create a bridge for the physical interface eth0 as follows:
Code:

config_eth0=null

#Configure TUN/TAP interface
tuntap_tap0="tap"

# tap0 defined empty to avoid DHCP being run for their configuration
config_tap0=null

# Configure network bridge
config_br0="198.xx.xx.225 netmask 255.255.255.0 brd 198.xx.xx.255"
brctl_br0="setfd 0
sethello 10
stp off"
routes_br0="default via 198.xx.xx.1"
mac_br0="aa:bb:cc:dd:ee:ff"
bridge_br0="eth0" # add all interfaces to bridge, usually use one TUN/TAP interface for one Vritual Machine. In this example we have one VM.

Now when I run
Code:

$ sudo /etc/init.d/net.br0
* Bringing up interface br0
*   Creating bridge br0
add bridge failed: package not installed
* ERROR: net.br0 failed to start


I am not sure what package it is referring to. I already have net-misc/bridge-utils installed.
----------
_________________
Unanswered Post Initiative:
https://forums.gentoo.org/viewtopic.php?t=119906
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43753
Location: 56N 3W

PostPosted: Tue Oct 06, 2015 6:16 pm    Post subject: Reply with quote

dustfinger,

Its an entry in /etc/init.d/ its looking for.
Code:
config_br0="198.xx.xx.225 netmask 255.255.255.0 brd 198.xx.xx.255"
is a static setup.

Much as you now have an net.eth0 -> net.lo in /etc/init.d/, you need a net.br0 -> net.lo symlink
You should remove the net.eth0 -> net.lo symlink meanwhile.

You also need to swap entries in your default runlevel if you want br0 to start automatically.

--- edit ---

You have it
Code:
$ sudo /etc/init.d/net.br0
Hopefully you had start on the end of that.

Try
Code:
$ sudo /etc/init.d/net.br0 -v start
The -v is verbose made.
You need bridge support in your kernel too.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
dustfinger
Guru
Guru


Joined: 15 Aug 2004
Posts: 449

PostPosted: Sun Oct 11, 2015 2:15 pm    Post subject: Reply with quote

Hi NeddySeagoon,

I have not been able to respond until now because I have been burdened with other work. I will be back to working on this issue tonight though. I wanted to post an update because I have changed my configuration somewhat and made some progress.

Please see my network diagram which illustrates what my plan is.

The following is my current /etc/conf.d/net configuration:
Code:
config_eth0=null

vlans_eth0="2 3"

eth0_valn2_name="eth0.2"
config_eth0_2="198.27.xx.25" netmask 255.255.255.0 brd 198.27.xx.255
routes_eth0_2="default via 192.27.xx.1"

eth0_valn3_name="eth0.3"
config_eth0_3="192.95.xx.33" netmask 255.255.255.0 brd 192.95.xx.255
mac_eth0_3="00:50:56:xx:yy:zz" # virtual mac assigned by OVH to 192.95.xx.33
dns_servers="8.8.8.8 8.8.4.4" #google's dns servers


I got rid of the /etc/init.d/net.br0 and the vlan links in /etc/init.d/. I have found that the vlans are automatically created in /proc/net/vlan. For the moment I am trying to get the vlan up without worrying about bridging.

When I run /etc/init.d/net eth0.2 and eth0.3 are correctly assigned ip addresses and I can ping them from the host, but I cannot ping them from outside the host, nor can I ping google.com. I am now wondering if making a vlan aware bridge would help. Someone on IRC told me that he believed ovh security will not allow me to use vlans. I am wanting to give it a bit more of a try though because it would be really nice to use vlans.

Note, that earlier I did not have a virtual mac address for my second static ip. I have now configured this with ovh: http://help.ovh.com/DedieMac and assign that interface with the virtual mac address.

Today I am going to be mostly busy with thanks giving dinner, but late tonight I will get back to working on this. Thank you for your responses so far. I really appreciate it. I know very little about networking and so there has been a large learning curve for me to understand what vlans and bridges are really used for and why I need them etc.

Sincerely,

dustfinger
_________________
Unanswered Post Initiative:
https://forums.gentoo.org/viewtopic.php?t=119906
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43753
Location: 56N 3W

PostPosted: Sun Oct 11, 2015 7:52 pm    Post subject: Reply with quote

dustfinger,

OVH - Oh dear.
I left them a while ago for Hetzner. I got better hardware - more of everything for the same cost. The support is much better too.
I hope you do not use the OVH Gentoo offering.

IPv6 for your KVMs is straight forward.

Is there a reason why you appear to be avoiding virt-manager?
The command line is doing it the hard way.

For firewalling, you can set up a bridge with no real hardware assigned, give it a IP address then forward packets to it with IPtables.
It will bridge the filtered packets.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
merky1
n00b
n00b


Joined: 22 Apr 2003
Posts: 51

PostPosted: Mon Oct 12, 2015 12:17 pm    Post subject: Reply with quote

First bridges are not like routers. They are more like switches. Basically, they provide a virtual bus for the virtual network adapters to "plug" into. They allow you to do things like create bridges that have no physical adapters (host only isolated networks) and have bridges connected to a physical adapter that does not use an address.

On my system I have a bridge layout like this :
Code:
br0             8000.18a905775794       yes             enp2s0f0
br1             8000.18a905775796       yes             enp2s0f1
                                                        vnet0
                                                        vnet1
                                                        vnet2
                                                        vnet3
                                                        vnet4
                                                        vnet5
                                                        vnet6
                                                        vnet7

br0 is the primary management network, with an address on my "management" vlan. br1 is connected to my "virtualization" vlan, which is where all the vnet / virtual machines connect.
_________________
ooo000 WoooHooo 000ooo
Back to top
View user's profile Send private message
dustfinger
Guru
Guru


Joined: 15 Aug 2004
Posts: 449

PostPosted: Tue Oct 13, 2015 8:08 am    Post subject: Reply with quote

NeddySeagoon wrote:
OVH - Oh dear.
I left them a while ago for Hetzner. I got better hardware - more of everything for the same cost. The support is much better too.
I hope you do not use the OVH Gentoo offering.
I keep hearing that people are not happy with OVH. I did not use their Gentoo template; the first thing I did was ordered a usb drive, downloaded a live cd and installed Gentoo from scratch. I am going to finish the process of setting this server up with OVH, since I am their right now. I will checkout Hezner though as a future option since you recommend it.
NeddySeagoon wrote:

Is there a reason why you appear to be avoiding virt-manager?
The command line is doing it the hard way.
I am trying to expand my understanding. I have been doing a whole lot of man page reading and experimenting while setting up this server.

NeeddySeagoon wrote:
For firewalling, you can set up a bridge with no real hardware assigned, give it a IP address then forward packets to it with IPtables.
It will bridge the filtered packets.
Perfect, that is what I will do!

merky1: Is vnetx the same thing as vlanx? Or is vnet something slightly different? Is vnetx just the interfaces for each virtual machine? I have not yet created my virtual machine, i was trying to solve these networking problems first.

Sincerely,

dustfinger

-- EDIT --

NeddySeagoon, thank you for the tip about using ipv6. That also lead me to libvirt for managing qemu and OpenVSwitch. I have only been reading up on all of this so far, but I will start actually installing and configuring everything tomorrow night.
----------
_________________
Unanswered Post Initiative:
https://forums.gentoo.org/viewtopic.php?t=119906
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43753
Location: 56N 3W

PostPosted: Tue Oct 13, 2015 5:28 pm    Post subject: Reply with quote

dustfinger,

OVH give you a rescue system. Its a net booted debian based system I think.
That's how I installed my Gentoo at OVH.

They used to provide a way to boot your install into QEMU, so you you could get the console.
Unfortunately, I got Intel real hardware and QEMU pretended to be AMD, so with everything built -march=native, QEMU failed for other reasons.

The OVH forums are helpful. Some of the OVH support staff post there but official support is fairly limited, to put it kindly.
My OVH install is still alive and well as a KVM on my Hetzner box.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
dustfinger
Guru
Guru


Joined: 15 Aug 2004
Posts: 449

PostPosted: Thu Oct 15, 2015 3:27 am    Post subject: Reply with quote

Hi NeddySeagoon,

I never got a rescue system with ovh. I did get an IPMI (I have always worried about the security of this) which gives me Keyboard Video Monitor access. I used that in conjunction with a usb drive to install gentoo from a live cd.

I am currently considering one of the following options to move forward with:
1. to use libvirt/virsh to manage my qemu kvms and OpenVSwitch to manage my network
2. To use OpenStack to manage everything. The downside here is that I have only one physical host. It does have 12 cpu and 64 GB of RAM though

I think that option 1 would do well for me, but I if I do get more hosts in the future, I don't imagine that it would be very easy to switch to openstack at that point. Whereas, if I just start with openstack now, it might be overkill, but it might also be easy to expand to additional hosts as I acquire and require them.

Sincerely,

dustfinger
_________________
Unanswered Post Initiative:
https://forums.gentoo.org/viewtopic.php?t=119906
Back to top
View user's profile Send private message
merky1
n00b
n00b


Joined: 22 Apr 2003
Posts: 51

PostPosted: Tue Nov 10, 2015 12:22 am    Post subject: Reply with quote

dustfinger wrote:


merky1: Is vnetx the same thing as vlanx? Or is vnet something slightly different? Is vnetx just the interfaces for each virtual machine? I have not yet created my virtual machine, i was trying to solve these networking problems first.



vnet[x] is not the same as vlanx. It is the virtualized NIC's connection to the hosts bridge. Basically uses a TUN/TAP interface.

I would highly recommend you stay away from the OpenStack stuffs until you grow beyond 5 hosts. Libvirt/Virtual Manager will do you well until you decide to expose things externally / for money.
_________________
ooo000 WoooHooo 000ooo
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum