Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Testing L2 routing with BATMAN
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3104

PostPosted: Wed Mar 08, 2023 2:03 pm    Post subject: Testing L2 routing with BATMAN Reply with quote

Been looking for things that could help deal with small networks easily, and wanted to share one of the things I've ran across.
BATMAN. This name pops up in the kernel config, so most of people here probably know it exists, but I doubt anyone tried using it, even though it seems pretty cool. And I wanted to give it a shot.

BATMAN is a protocol for creating a self-organizing, self-healing mesh. It runs in kernel, and batman interfaces act as both, entry point to the overlay network and routers for their batman neighbours.

Don't have a lot of hardware lying around me right now, so went with namespaces, veth, iperf and top on a quad-core laptop.
Tests done:
iperf on veth pair (no batman)
cross-talk between batman and veth (the flock of 1 batman doesn't fly, must be a least a pair)
iperf on 2 node batman pair
iperf on 3 node batman loop
iperf on 3 node batman chain

varying number of hops in 16-node long loop:
1 hop
5 hops
8 hops

Also:
8 hops across across 16-node ladder (loop 1-2-3-4 ...15-16-1 + shortcuts 2-16;3-15;4-14 and so on. 1 and 9 are the test points and all shortcuts are in the wrong direction)


A simple script (which admittedly could have been written in a better way) used to setup the loop fast:
Code:
# cat battest_setup
#! /bin/bash

for i in $( seq 1 $1 )
do      ip netns add ns${i}
done

j=$1
ip link add dev eth${j}a type veth peer name eth${j}b
for i in $( seq 1 $(($1-1)) )
do      ip link add dev eth${i}a type veth peer name eth${i}b
        ip link set eth${i}a netns ns${i}
        ip link set eth${j}b netns ns${i}
        ip netns exec ns${i} ip link add name bat${i} type batadv
        ip netns exec ns${i} ip link set bat${i} up
        ip netns exec ns${i} ip a add 10.1.1.${i}/24 dev bat${i}
        ip netns exec ns${i} ip link set dev eth${i}a master bat${i} up
        ip netns exec ns${i} ip link set dev eth${j}b master bat${i} up

        j=${i}
done

i=$1
ip link set eth${i}a netns ns${i}
ip link set eth${j}b netns ns${i}
ip netns exec ns${i} ip link add name bat${i} type batadv
ip netns exec ns${i} ip link set bat${i} up
ip netns exec ns${i} ip a add 10.1.1.${i}/24 dev bat${i}
ip netns exec ns${i} ip link set dev eth${i}a master bat${i} up
ip netns exec ns${i} ip link set dev eth${j}b master bat${i} up


Iperf test results:
Code:

### Direct connect via veth:
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.2 port 40528 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  8.76 GBytes  75.2 Gbits/sec    0    523 KBytes
[  5]   1.00-2.00   sec  8.82 GBytes  75.7 Gbits/sec    0    837 KBytes
[  5]   2.00-3.00   sec  8.80 GBytes  75.6 Gbits/sec    0    882 KBytes
[  5]   3.00-4.00   sec  8.81 GBytes  75.7 Gbits/sec    0   1.05 MBytes
[  5]   4.00-5.00   sec  8.87 GBytes  76.2 Gbits/sec    0   1.10 MBytes
[  5]   5.00-6.00   sec  8.53 GBytes  73.2 Gbits/sec    0   1.62 MBytes
[  5]   6.00-7.00   sec  8.49 GBytes  72.9 Gbits/sec    0   1.70 MBytes
[  5]   7.00-8.00   sec  8.47 GBytes  72.8 Gbits/sec    0   1.87 MBytes
[  5]   8.00-9.00   sec  8.79 GBytes  75.5 Gbits/sec    0   1.87 MBytes
[  5]   9.00-10.00  sec  8.80 GBytes  75.6 Gbits/sec    0   1.96 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  87.1 GBytes  74.8 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  2.00 GBytes  1.72 Gbits/sec                  receiver


### BATMAN 1 hop
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.2 port 56408 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   520 MBytes  4.36 Gbits/sec   32    392 KBytes
[  5]   1.00-2.00   sec   519 MBytes  4.35 Gbits/sec  409    501 KBytes
[  5]   2.00-3.00   sec   525 MBytes  4.41 Gbits/sec  548    369 KBytes
[  5]   3.00-4.00   sec   527 MBytes  4.42 Gbits/sec  288    445 KBytes
[  5]   4.00-5.00   sec   529 MBytes  4.44 Gbits/sec   86    492 KBytes
[  5]   5.00-6.00   sec   420 MBytes  3.53 Gbits/sec  115    380 KBytes
[  5]   6.00-7.00   sec   523 MBytes  4.38 Gbits/sec  362    269 KBytes
[  5]   7.00-8.00   sec   529 MBytes  4.44 Gbits/sec  301    402 KBytes
[  5]   8.00-9.00   sec   534 MBytes  4.48 Gbits/sec  586    293 KBytes
[  5]   9.00-10.00  sec   527 MBytes  4.41 Gbits/sec  216    329 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  5.03 GBytes  4.32 Gbits/sec  2943             sender
[  5]   0.00-10.00  sec  2.00 GBytes  1.72 Gbits/sec                  receiver


### BATMAN 1 hop; loop of 3
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.3 port 50604 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   432 MBytes  3.62 Gbits/sec  753    222 KBytes
[  5]   1.00-2.00   sec   438 MBytes  3.67 Gbits/sec  847    197 KBytes
[  5]   2.00-3.00   sec   437 MBytes  3.67 Gbits/sec  375    283 KBytes
[  5]   3.00-4.00   sec   261 MBytes  2.19 Gbits/sec  586    297 KBytes
[  5]   4.00-5.00   sec   350 MBytes  2.94 Gbits/sec  696    417 KBytes
[  5]   5.00-6.00   sec   348 MBytes  2.92 Gbits/sec  446    313 KBytes
[  5]   6.00-7.00   sec   436 MBytes  3.66 Gbits/sec  258    428 KBytes
[  5]   7.00-8.00   sec   279 MBytes  2.34 Gbits/sec  335    346 KBytes
[  5]   8.00-9.00   sec   338 MBytes  2.83 Gbits/sec  311   1.20 MBytes
[  5]   9.00-10.00  sec   325 MBytes  2.73 Gbits/sec  778   1.16 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  3.56 GBytes  3.06 Gbits/sec  5385             sender
[  5]   0.00-10.00  sec  2.00 GBytes  1.72 Gbits/sec                  receiver



### BATMAN 2 hops; chain of 3
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.3 port 43520 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   428 MBytes  3.59 Gbits/sec  382    243 KBytes
[  5]   1.00-2.00   sec   436 MBytes  3.65 Gbits/sec  465    322 KBytes
[  5]   2.00-3.00   sec   346 MBytes  2.91 Gbits/sec  364    226 KBytes
[  5]   3.00-4.00   sec   432 MBytes  3.63 Gbits/sec  321    300 KBytes
[  5]   4.00-5.00   sec   440 MBytes  3.69 Gbits/sec  147    301 KBytes
[  5]   5.00-6.00   sec   355 MBytes  2.97 Gbits/sec   22    253 KBytes
[  5]   6.00-7.00   sec   348 MBytes  2.93 Gbits/sec  788    267 KBytes
[  5]   7.00-8.00   sec   432 MBytes  3.62 Gbits/sec  600    286 KBytes
[  5]   8.00-9.00   sec   445 MBytes  3.73 Gbits/sec   45    286 KBytes
[  5]   9.00-10.00  sec   445 MBytes  3.73 Gbits/sec  185    464 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.01 GBytes  3.45 Gbits/sec  3319             sender
[  5]   0.00-10.00  sec  2.00 GBytes  1.72 Gbits/sec                  receiver


### BATMAN 1/16 hops
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.2 port 59162 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   520 MBytes  4.36 Gbits/sec  255    300 KBytes
[  5]   1.00-2.00   sec   514 MBytes  4.31 Gbits/sec   41    402 KBytes
[  5]   2.00-3.00   sec   523 MBytes  4.39 Gbits/sec  979    284 KBytes
[  5]   3.00-4.00   sec   530 MBytes  4.44 Gbits/sec  460    433 KBytes
[  5]   4.00-5.00   sec   539 MBytes  4.53 Gbits/sec  323    457 KBytes
[  5]   5.00-6.00   sec   535 MBytes  4.48 Gbits/sec  134    457 KBytes
[  5]   6.00-7.00   sec   532 MBytes  4.47 Gbits/sec   99    467 KBytes
[  5]   7.00-8.00   sec   534 MBytes  4.47 Gbits/sec   34    522 KBytes
[  5]   8.00-9.00   sec   315 MBytes  2.64 Gbits/sec  353    199 KBytes
[  5]   9.00-10.00  sec   422 MBytes  3.55 Gbits/sec    6    393 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.85 GBytes  4.16 Gbits/sec  2684             sender
[  5]   0.00-10.00  sec  2.00 GBytes  1.72 Gbits/sec                  receiver


### BATMAN 5/16 hops
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.6 port 52260 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   258 MBytes  2.16 Gbits/sec  147    803 KBytes
[  5]   1.00-2.00   sec   244 MBytes  2.04 Gbits/sec  573    246 KBytes
[  5]   2.00-3.00   sec   165 MBytes  1.38 Gbits/sec   44   1.56 MBytes
[  5]   3.00-4.00   sec   258 MBytes  2.16 Gbits/sec   40   1.93 MBytes
[  5]   4.00-5.00   sec   206 MBytes  1.73 Gbits/sec  283    553 KBytes
[  5]   5.00-6.00   sec   256 MBytes  2.15 Gbits/sec    1   1.72 MBytes
[  5]   6.00-7.00   sec   202 MBytes  1.70 Gbits/sec  188    980 KBytes
[  5]   7.00-8.00   sec   258 MBytes  2.16 Gbits/sec    0   1.03 MBytes
[  5]   8.00-9.00   sec   256 MBytes  2.15 Gbits/sec   45   1.08 MBytes
[  5]   9.00-10.00  sec   155 MBytes  1.30 Gbits/sec  247    700 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.20 GBytes  1.89 Gbits/sec  1568             sender
[  5]   0.00-10.00  sec  2.00 GBytes  1.72 Gbits/sec                  receiver


### BATMAN 8/16 hops in the ring
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.9 port 58916 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   180 MBytes  1.51 Gbits/sec  457    273 KBytes
[  5]   1.00-2.00   sec   178 MBytes  1.50 Gbits/sec  208    491 KBytes
[  5]   2.00-3.00   sec   179 MBytes  1.50 Gbits/sec   68    520 KBytes
[  5]   3.00-4.00   sec   177 MBytes  1.49 Gbits/sec    6    468 KBytes
[  5]   4.00-5.00   sec   142 MBytes  1.20 Gbits/sec   95    436 KBytes
[  5]   5.00-6.00   sec   180 MBytes  1.51 Gbits/sec   50    578 KBytes
[  5]   6.00-7.00   sec   143 MBytes  1.20 Gbits/sec  105    795 KBytes
[  5]   7.00-8.00   sec   179 MBytes  1.50 Gbits/sec  232    574 KBytes
[  5]   8.00-9.00   sec   180 MBytes  1.51 Gbits/sec   38    382 KBytes
[  5]   9.00-10.00  sec   180 MBytes  1.51 Gbits/sec    2    438 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.68 GBytes  1.44 Gbits/sec  1261             sender


### BATMAN 8/16 hops on ladder shape
Connecting to host 10.1.1.1, port 5201
[  5] local 10.1.1.9 port 34524 connected to 10.1.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   151 MBytes  1.26 Gbits/sec  220    414 KBytes
[  5]   1.00-2.00   sec   147 MBytes  1.23 Gbits/sec  1342    383 KBytes
[  5]   2.00-3.00   sec   155 MBytes  1.30 Gbits/sec  530    298 KBytes
[  5]   3.00-4.00   sec   162 MBytes  1.36 Gbits/sec  266    402 KBytes
[  5]   4.00-5.00   sec   161 MBytes  1.35 Gbits/sec  404    300 KBytes
[  5]   5.00-6.00   sec   160 MBytes  1.34 Gbits/sec    3    395 KBytes
[  5]   6.00-7.00   sec   158 MBytes  1.32 Gbits/sec    6    329 KBytes
[  5]   7.00-8.00   sec   160 MBytes  1.35 Gbits/sec    4    355 KBytes
[  5]   8.00-9.00   sec   118 MBytes   991 Mbits/sec   14    286 KBytes
[  5]   9.00-10.00  sec   158 MBytes  1.32 Gbits/sec    3    270 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.49 GBytes  1.28 Gbits/sec  2792             sender
[  5]   0.00-10.00  sec  1.49 GBytes  1.28 Gbits/sec                  receiver


The bigger the network, the slower it gets, not very surprising, however the connections not on the path also influence transfer speed (the ladder was slower than ring, 3 node loop was slower than a pair)
It seems that speed on the long chains degrades due to latency, since running iperf did not cause big load spikes. Also, while BATMAN is capable of running on wires, it's designed for highly dynamic networks, like ad-hoc wifi with mobile nodes. Wifi is WAY slower than veth, so the processing latency might not be noticeable in its intended use case scenario.
Could also be some attempt at bandwidth sharing.

Although I haven't found explicit information about applicability, I noticed e.g. multicast optimizations which appear to target networks with a few dozens of nodes.
The longest chain that works by default is 32 nodes, batctl does not seem to have any way to change it, throughput went down do 50MBps though.

Bonus: there are 2 shortest paths in my ring test and batman picks them more or less randomly:
Code:
### BATMAN changing active chain
 ~ # batctl meshif bat9 tr 10.1.1.1
traceroute to 10.1.1.1 (46:52:77:89:25:b4), 50 hops max, 20 byte packets
 1: a2:48:a1:d3:33:99  0.028 ms  0.002 ms  0.001 ms
 2: fa:a1:d3:6b:be:30  0.002 ms  0.002 ms  0.002 ms
 3: e6:8c:45:d3:4c:5b  0.001 ms  0.001 ms  0.001 ms
 4: ba:6c:d8:53:bb:8c  0.002 ms  0.001 ms  0.001 ms
 5: ce:66:9c:e2:2b:88  0.001 ms  0.001 ms  0.001 ms
 6: 56:d9:d4:9c:19:bc  0.001 ms  0.001 ms  0.001 ms
 7: be:ae:36:15:a9:31  0.001 ms  0.001 ms  0.001 ms
 8: 46:52:77:89:25:b4  0.002 ms  0.001 ms  0.001 ms
 ~ # batctl meshif bat9 tr 10.1.1.1
traceroute to 10.1.1.1 (46:52:77:89:25:b4), 50 hops max, 20 byte packets
 1: 76:c6:91:b7:3e:be  0.006 ms  0.002 ms  0.002 ms
 2: 92:8c:8e:9f:26:9a  0.002 ms  0.001 ms  0.001 ms
 3: a6:85:a4:2a:bd:7e  0.002 ms  0.002 ms  0.001 ms
 4: 6a:4b:05:1c:23:0e  0.002 ms  0.001 ms  0.001 ms
 5: 82:8f:04:90:28:1a  0.002 ms  0.002 ms  0.001 ms
 6: 7a:b1:3a:98:c7:a7  0.002 ms  0.001 ms  0.001 ms
 7: 62:1d:2f:dc:5e:97  0.002 ms  0.002 ms  0.001 ms
 8: 46:52:77:89:25:b4  0.002 ms  0.002 ms  0.001 ms
 ~ # batctl meshif bat9 tr 10.1.1.1
traceroute to 10.1.1.1 (46:52:77:89:25:b4), 50 hops max, 20 byte packets
 1: a2:48:a1:d3:33:99  0.006 ms  0.002 ms  0.001 ms
 2: fa:a1:d3:6b:be:30  0.002 ms  0.001 ms  0.016 ms
 3: e6:8c:45:d3:4c:5b  0.002 ms  0.001 ms  0.001 ms
 4: ba:6c:d8:53:bb:8c  0.001 ms  0.002 ms  0.001 ms
 5: ce:66:9c:e2:2b:88  0.001 ms  0.001 ms  0.001 ms
 6: 56:d9:d4:9c:19:bc  0.001 ms  0.001 ms  0.001 ms
 7: be:ae:36:15:a9:31  0.022 ms  0.001 ms  0.001 ms
 8: 46:52:77:89:25:b4  0.002 ms  0.001 ms  0.001 ms



Wrapping up, it exists, it is simple to use, if anyone runs it in real life I'm curious how it performs.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum