| View previous topic :: View next topic |
| Author |
Message |
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
 Posted: Fri Feb 27, 2015 7:39 pm Post subject: libteam problems, has anyone tried NIC teaming ? |
 |
|
I have a souple macines connected through level good 1 switch.
One of those is used as a file server and I have a couple other workstations that could use a bit fatter pipe, so I decided to try card teaming.
I installed on server and one worksation pairt of Intel's cheap but good Gig-E cards ( Intel Gigabit CT Desktop Adapter - based on 82574L).
New NIC's work great. Far better than old crappy Realtek.
But now I tried to team them in a pair with teamd and this fails.
I have teaming support compiled-in kernel, with all the available runners.
When I start teamd in debug mode, it read the config and then exits with message (I'm paraphrasing):
"no loadbalace" runner available.
I tried packaged examples with other runners, like roundrobin etc, but response was same: no "XXX runner found"
BTW: I run systemd with networking set up so that newtorkd renames NICs from default kernel names to lan0 and lan1. I don't have corresponding network files set, so NICS are just named lan0/1 and their NIC is set, nothing more ( no IP setting, no DHCP etc).
Has anyone tried this ?
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
 |
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3134
|
| Posted: Fri Feb 27, 2015 8:32 pm Post subject: |
|
|
show us your code*
No, really, it will e easier to hunt bugs when we know what you have actually done.
Also, I saw both, teaming and bonding in kernel. Perhaps you compiled teaming but attempt to use bonding instead? Based on brief search, those are not likely to be compatable
*how do I make it blink?  |
|
| Back to top |
|
|
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
| Posted: Fri Feb 27, 2015 8:38 pm Post subject: |
|
|
| szatox wrote: | show us your code*
No, really, it will e easier to hunt bugs when we know what you have actually done.
Also, I saw both, teaming and bonding in kernel. Perhaps you compiled teaming but attempt to use bonding instead? Based on brief search, those are not likely to be compatable
*how do I make it blink? |
No need. I think I found the culprit. I now tried same on workstation and it works.
It seems that I f**ed kernel installation.
I've found that I don't have teaming support, so I compiled it in. But I forgot that I have setup grub to boot from USB stick that I have reworked so that it was plug into internal MoBo USB connector.
So I installed kenel on boot partition of ordinary disks, which isn't used on boot.
Methinks this will work now...
Thanks for your time and sorry for inconvenience.
Murphy's law - answer will become obvious immediatelly after you give up... ;o)
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3134
|
| Posted: Fri Feb 27, 2015 8:52 pm Post subject: |
|
|
Yeah I love Murphy's law on separate /boot too. Whenever I wanted to upgrade kernel I'd forget to mount it  |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54234
Location: 56N 3W
|
| Posted: Fri Feb 27, 2015 9:00 pm Post subject: |
|
|
szatox,
is your friend there :) ... I have a separate boot everywhere too.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3134
|
| Posted: Sat Feb 28, 2015 10:00 am Post subject: |
|
|
actually the whole point there was about installing kernel to a non-bootable (virtual, not-existent before boot) device rather than booting wrong kernel. So I had to fix the mistake and reboot again. And sometimes again, if I was distracted with some other stuff
Enabling auto-mounting /boot upon kernel compilation did the trick though |
|
| Back to top |
|
|
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
| Posted: Sat Feb 28, 2015 4:47 pm Post subject: |
|
|
update:
I have gone through quite a fewe headaches to get it to kind of work.
But it worked quite weirldy. For example, I could never see the machine at the other end of the pipe.
But traffic relaying or adressing some other machine further away worked fine.
I thought that main culprit is teamd whic shiwthes all MAC adresses of all teamed cards as wel as team master to the same value.
But after playing some more with individual NICs I came to N-th proof that systemd's networkd is utter piece of crap.
I deactivated it and now I can get network under control, so now I will try again...
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3134
|
| Posted: Sat Feb 28, 2015 9:58 pm Post subject: |
|
|
| Quote: | | I thought that main culprit is teamd whic shiwthes all MAC adresses of all teamed cards as wel as team master to the same value |
this might be a problem if you have a single switch. It will not be able to determine which port is assocated with your MAC address, particularly at higher bitrates. No idea what modes teaming supports, but if keeping MAC of the physical NIC is possible (and binding a particular conection to that MAC) it will be much healthier for your network.
Anyway, let us know how it's doing. Any chance for performance test? |
|
| Back to top |
|
|
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
| Posted: Sun Mar 01, 2015 12:26 pm Post subject: |
|
|
| szatox wrote: | | Quote: | | I thought that main culprit is teamd whic shiwthes all MAC adresses of all teamed cards as wel as team master to the same value |
this might be a problem if you have a single switch. It will not be able to determine which port is assocated with your MAC address, particularly at higher bitrates. No idea what modes teaming supports, but if keeping MAC of the physical NIC is possible (and binding a particular conection to that MAC) it will be much healthier for your network.
Anyway, let us know how it's doing. Any chance for performance test? |
Yes, but that was a reason why I went for teaming instead of bonding. They said teaming should work with ordianry switches just fine.
WRT to final report, it'll have to wait. I touched networking on systemd and as always, was rewarded with shower of courious fu**ups and anomalies.
It feels almost like Star Trek - "Captain, unusual gravimetric phenomenon ahead". 
After gong for git-systemd, I am now trying latest available gentoo-sources.
Maybe this will fix or at least mend it.
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
|
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
| Posted: Mon Mar 02, 2015 6:07 am Post subject: |
|
|
I have just managed to get it to work.
COnfiguration:
Server Phenom 955 + 8G RAM + 2x Intel NICs ( serves also as router/firewall). It has 3 NICs -- 1 Realtek onboard + 2 Intel CT NICs. Realtek is connected to the net, two Intels are teamed and connected to 24-port switch HP1410-24G ( highest fabric throughput submodel J9561A). Server has 3 x 2TB HDD ( SAMSUNG SpinPoint F3 EG ) in RAID5.
Linear read of 1GB block from RAID5 locally gets me 180+ to 210 MiB/s transfer.
Workstation is similar Phenom machine with a pair of Intel's CT adapters, connected to the same switch.
I have nfsv4 shares on server. 1G read of o file on filesystem on the share through one NIC gets me around 80MiB/s.
Same read through teamed pair gets me 125+ MiB/s, without any parameter optimisations.
I have a feeling I could lift that considerably once I add a disk or two to the RAID and tweak LAN parameters.
Dodn't have the time yet to check how much does this disturb the switch.
I am using simple roundrobin mode within team driver. Will check if loadbalance or lacp mode change things consifderably.
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
|
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
| Posted: Mon Mar 02, 2015 2:47 pm Post subject: |
|
|
I've just tried iperf3 NIC benchamrking tool and it gives me very low figures for bandwidth - around a couple of MEGAbits per second , about 1/1000 of what is actually avaialble.
It seems that team device confuses it.
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
|
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
| Posted: Wed Mar 11, 2015 8:02 pm Post subject: |
|
|
One more problem:
Traffic between a team and one simple "non-teamed" NIC is ( thorough otherwise good L1 switch) horribly slow. I get 1/10 of theoretically available bandwith or less.
I am using round-robin runner. Haven't trtied with lacp yet, though.
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3134
|
| Posted: Wed Mar 11, 2015 9:07 pm Post subject: |
|
|
I don't know how it's implemented in team driver, but some modes (including rr) work better when network switches know what's going on and cooperate. E.g. RR typicaly sets the same MAC on all interfaces which is fine when every NIC is connected to another switch (And switches are not connected) so you have 2 independent paths and addresses don't conflict because from switches' points of view they are uniqe.
| Code: | host A
/ \
NIC1a NIC2a
| |
Switch1 Switch2 (note: in this case dumb switches are fine. If they were connected, they would not be)
| |
NIC1b NIC2b
\ /
host B |
If you connect 2 NICs with the same MAC to a single switch things are likely to get weird. You can never be sure what such a switch will do, unless it's smart enough to do round robin itself - and configured to do so.
What happens if you unplug one path? Like in keeping round robin configured on 2 NICs, but only 1 of those connected to the switch with the other physical link dead? |
|
| Back to top |
|
|
Brane2
n00b
Joined: 03 Jul 2011
Posts: 44
|
| Posted: Thu Mar 12, 2015 10:01 am Post subject: |
|
|
Switch seems to be handling traffic just fine. It's 24-way switchst from HP, designed to handle mximal throughput on all ports simulatnepously.
Communication of two teams through that switch goes well above 1Gb/s.
Team driver configures the same MAC on both cards, but no IP - that is, except link-local IPv6 ( fe80:...)
_________________
On the journey of life I chose the psycho path... |
|
| Back to top |
|
|
|
Other Things Gentoo
