one way communication with kvm+macvlan

[_______0]

hi,

host macvlan
guest macvtap

I can ssh from guest to host but no the other way around.

Anyone know the trick for this?

thanks.

[John R. Graham]

Did you set up both of these (virtual?) machines?

- John

[_______0]

macvlan (host is not virtual)
mavtap (guest is vm)

I don't understand your question. But the answer is of course it's set up. How else would I be able to ssh from guest to host?

To clarify my confusion, what exactly and which part do you mean by 'set up'?

[John R. Graham]

Sorry; wn a little bit more detail, what I meant was, did you install Gentoo on both of these machines and do you have control over both of them?

- John

[papahuhn]

hi,

host macvlan
guest macvtap

I can ssh from guest to host but no the other way around.

Anyone know the trick for this?

thanks.

If you have established an ssh session, then this is already two way communication, and macvlan/macvtap is irrelevant. How about iptables?

[_______0]

... How about iptables?

I am all ears.

There was a mistake on my post. The connection was established over a router so the bridge mod in macvlan wasn't doing what's supposed to do.

Is it possible to have two nics have a conversation with each other on the same computer with iptables? What part gets involved in this, routing, iptables?

To be honest the popular bridging method I don't see very elegant as it kills flexibility.

I've been reading some documentation and this huge thread, http://markmail.org/message/vudd7mvq3c5ze3j6 , but the they are quite incloncusive.

I need VM's to be able to talk to each other as well to host without a router being involved. ssh'ing is better than bloated desktop, sometimes the kvm hangs and my mouse and keyboard get stuck inside and I have to reboot!!!

macvlan has three modes one which is bridge which is independent from external router/switch.

One guy claims it works, http://lists.gnu.org/archive/html/qemu- ... 02687.html ,

Right, but if I set IP(eth0) == IP(macvlan0), I'm able to communicate
between macvlan0 and mactapX, thus between guest and host. Just
re-checked here, still works (after resolving the usual MAC address mess
I caused by configuring manually).

Actually even with traditional method using brctl there's gotta be an ip in common where both interface are able to talk. The routing part perhaps?

I think I am not doing this properly, just assigning an ip to macvlans and setting them in bridge mode is not enough? Thinking about it makes sense an extra framework for communication.

thanks

[papahuhn]

There was a mistake on my post. The connection was established over a router so the bridge mod in macvlan wasn't doing what's supposed to do.

Ok, this changes things. I just googled that your problem is "defined behaviour": http://wiki.libvirt.org/page/Guest_can_ ... _(macvtap)

I can't help you with this by other means than googling, sorry.

Actually even with traditional method using brctl there's gotta be an ip in common where both interface are able to talk. The routing part perhaps?

With a traditional bridge, the host needs a route on that bridge towards the networks that are configured inside the VMs - typically by assigning br0 an IP address from those networks.

[_______0]

With a traditional bridge, the host needs a route on that bridge towards the networks that are configured inside the VMs - typically by assigning br0 an IP address from those networks.

can you show an example or some steps for this?

[papahuhn]

Code: Select all

root@host:~ ip a sh virbr1 | grep "inet "
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr1
root@host:~ brctl show
bridge name	bridge id		STP enabled	interfaces
virbr0		8000.047d7b9219be	no		eth0
virbr1		8000.fe54008b8897	no		vnet0
root@host:~ # vnet0 is the VMs host-side interface
root@host:~ # in this case, I don't bridge the VM with my physical network
root@host:~ # but with potential other VMs (currently not running)
root@host:~ route -n | grep virbr1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr1
root@host:~ ssh 192.168.122.124
root@192.168.122.124's password: 
Last login: Wed Jun 19 18:39:04 2013
[root@centos64 ~]# ip a sh eth0 | grep "inet "
    inet 192.168.122.124/24 brd 255.255.255.255 scope global eth0
[root@centos64 ~]# route -n
Kernel IP Routentabelle
Ziel            Router          Genmask         Flags Metric Ref    Use Iface
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
0.0.0.0         192.168.122.1   0.0.0.0         UG    0      0        0 eth0
[root@centos64 ~]# 

[_______0]

what about this claim that I found on two places?

Right, but if I set IP(eth0) == IP(macvlan0), I'm able to communicate
between macvlan0 and mactapX, thus between guest and host. Just
re-checked here, still works (after resolving the usual MAC address mess
I caused by configuring manually).

This implies something like this:

eth0 192.168.1.2
macvlan0 192.168.1.2

Is this correct? Even then, a place mentioned eth0 needed to have no routes, manually flushed, and guests have some weird thing about host's ip as route??

/me confused as hell.

[_______0]

anyone?

thanks

[papahuhn]

Nice, it works as described. You don't even need the IP on eth0, neither to set anything special within the VM.

[_______0]

Nice, it works as described. You don't even need the IP on eth0, neither to set anything special within the VM.

wait... Are you being sarcastic or something? What's working and how?? Plz tell me more!!

[papahuhn]

I'm not sarcastic. I just followed the tip from your link.
- Create a macvlan0@eth0
- Move IP and routes from eth0 to macvlan0
- Use macvtap@eth0 in bridge mode for your VMs (I use virt-manager for configuration).

[_______0]

I'm not sarcastic. I just followed the tip from your link.
- Move IP and routes from eth0 to macvlan0

:"((( works for everyone except me.

Can you show the steps for that?

Also, it's important that this works unplugged. Try ssh both ways host and guest without the cable and tell me if it works.

[papahuhn]

Also, it's important that this works unplugged. Try ssh both ways host and guest without the cable and tell me if it works.

Right, that doesn't work for me either. The macvlan and macvtap go into a LOWERLAYERDOWN state. While VMs still can communicate with each other, they cannot communicate with the host.

[_______0]

Also, it's important that this works unplugged. Try ssh both ways host and guest without the cable and tell me if it works.

Right, that doesn't work for me either. The macvlan and macvtap go into a LOWERLAYERDOWN state. While VMs still can communicate with each other, they cannot communicate with the host.

hence the suggestion of:

ip eth0 = macvlan

???

dunno...

[papahuhn]

The IPs don't matter here. The problem is with the link state. When the cable is unplugged, eth0's and macvlan's internal queueing discipline changes to noop [1].
In net/core/dev.c:dev_queue_xmit() a frame is then dropped by the noop qdisc, while macvlan transmission via macvlan_start_xmit() would be performed after.

So, the problem is not with receiving a frame from a VM but sending the response back to it on a practically DOWN interface.

Plugged:

Code: Select all

 2)               |        dev_queue_xmit() {
 2)               |          local_bh_disable() {
 2)   0.137 us    |            __local_bh_disable();
 2)   0.660 us    |          }
 2)   0.121 us    |          netdev_pick_tx();
 2)               |          _raw_spin_lock() {
 2)   0.085 us    |            add_preempt_count();
 2)   0.566 us    |          }
 2)               |          sch_direct_xmit() {
 2)               |            _raw_spin_unlock() {
 2)   0.077 us    |              sub_preempt_count();
 2)   0.612 us    |            }
 2)               |            dev_hard_start_xmit() {
 2)   0.155 us    |              netif_skb_features();
 2)               |              macvlan_start_xmit() {

Unplugged:

Code: Select all

 3)               |        dev_queue_xmit() {
 3)               |          local_bh_disable() {
 3)   0.120 us    |            __local_bh_disable();
 3)   0.638 us    |          }
 3)   0.118 us    |          netdev_pick_tx();
 3)               |          _raw_spin_lock() {
 3)   0.063 us    |            add_preempt_count();
 3)   0.591 us    |          }
 3)               |          noop_enqueue() {
 3)               |            kfree_skb() {
 3)               |              __kfree_skb() {
 3)               |                skb_release_head_state() {
 3)               |                  sock_wfree() {
 3)               |                    sock_def_write_space() {
 3)   0.058 us    |                      __rcu_read_lock();
 3)   0.060 us    |                      __rcu_read_unlock();
 3)   1.248 us    |                    }
 3)   1.944 us    |                  }
 3)   2.540 us    |                }
 3)               |                skb_release_data() {
 3)               |                  skb_free_head() {
 3)   0.190 us    |                    kfree();
 3)   0.679 us    |                  }
 3)   1.235 us    |                }
 3)   0.111 us    |                kmem_cache_free();
 3)   5.570 us    |              }
 3)   6.101 us    |            }
 3)   6.633 us    |          }

[1] http://ace-host.stuart.id.au/russell/fi ... h_noop.txt

[_______0]

mm... fuuu???


What about deleting OR instructing in net/core/dev.c NOT to change states?

Otherwise the next most simple setup would be to use VDE? To be honest the traditional bridging method somehow doesn't look very aesthetic.

thanks

[papahuhn]

There is no OR instruction, and the state-update code is somewhere else. I think you should ask the netdev mailing list.