tg3 kernel module not loaded at boot prior to net.eth0 setup

Message

beerbellyswan · Post by **beerbellyswan** » Tue Jun 19, 2018 9:15 pm

Hi folks - this has been a problem for a couple years and im just now aggravated enough to resolve it. When i boot, the tg3 kernel module is not being loaded early on so the net.eth0 setup fails. Message from boot.log:

Code: Select all

Executing: /lib64/rc/sh/openrc-run.sh /lib64/rc/sh/openrc-run.sh /etc/init.d/net.eth0 start
Bringing up interface eth0
  Caching network module dependencies need dbus
  Skipping module adsl due to missing program: /usr/sbin/adsl-start /usr/sbin/pppoe-start
  Skipping module br2684ctl due to missing program: br2684ctl
  Skipping module clip due to missing program: /usr/sbin/atmsigd
  Skipping module ethtool due to missing program: ethtool
  Skipping module netplugd due to missing program: /sbin/netplugd
  Skipping module ifplugd due to missing program: /usr/sbin/ifplugd
  Skipping module ipppd due to missing program: /usr/sbin/ipppd
  Skipping module iwconfig due to missing program: /sbin/iwconfig
  Skipping module firewalld due to missing program: firewall-cmd
  Skipping module pump due to missing program: /sbin/pump
  Loaded modules: apipa arping bonding l2tp tuntap bridge ccwgroup dummy hsr macvlan macchanger macnet wpa_supplicant ssidnet iproute2 pppd system vlan dhcpcd ip6rd ip6to4
  ip link set dev eth0 up
  ERROR: interface eth0 does not exist
  Ensure that you have loaded the correct kernel module for your hardware
ERROR: net.eth0 failed to start
ERROR: cannot start netmount as net.eth0 would not start

my workaround to this point is to simply run this command after logging in:

Code: Select all

rmmod tg3 && modprobe tg3 && ifconfig eth0

So tg3 is being loaded - but AFTER eth0 is being set up. I can't figure out why this is happening - any pointers?

krinn · Post by **krinn** » Tue Jun 19, 2018 10:41 pm

beerbellyswan wrote:
Code: Select all
rmmod tg3 && modprobe tg3 && ifconfig eth0
So tg3 is being loaded - but AFTER eth0 is being set up. I can't figure out why this is happening - any pointers?

Why you rmmod it?
you just need ifconfig eth0 to make it start or the reload sequence is need?

beerbellyswan · Post by **beerbellyswan** » Tue Jun 19, 2018 11:38 pm

@krinn - I remove it and reload it so that broadcom* starts using it. This is what it looks like when i first boot:

Code: Select all

office /home/rdswan # lsmod|grep tg3
tg3                   184320  0
libphy                 77824  1 tg3
office /home/rdswan #

And when it looks like that, i get this message when trying to start eth0:

Code: Select all

office /home/rdswan # /etc/init.d/net.eth0 start
 * Executing: /lib64/rc/sh/openrc-run.sh /lib64/rc/sh/openrc-run.sh /etc/init.d/net.eth0 start
 * Bringing up interface eth0
 *   Skipping module adsl due to missing program: /usr/sbin/adsl-start /usr/sbin/pppoe-start
 *   Skipping module br2684ctl due to missing program: br2684ctl
 *   Skipping module clip due to missing program: /usr/sbin/atmsigd
 *   Skipping module ethtool due to missing program: ethtool
 *   Skipping module netplugd due to missing program: /sbin/netplugd
 *   Skipping module ifplugd due to missing program: /usr/sbin/ifplugd
 *   Skipping module ipppd due to missing program: /usr/sbin/ipppd
 *   Skipping module iwconfig due to missing program: /sbin/iwconfig
 *   Skipping module firewalld due to missing program: firewall-cmd
 *   Skipping module pump due to missing program: /sbin/pump
 *   Loaded modules: apipa arping bonding l2tp tuntap bridge ccwgroup dummy hsr macvlan macchanger macnet wpa_supplicant ssidnet iproute2 pppd system vlan dhcpcd ip6rd ip6to4
 *   ip link set dev eth0 up
 *   ERROR: interface eth0 does not exist
 *   Ensure that you have loaded the correct kernel module for your hardware
 * ERROR: net.eth0 failed to start
office /home/rdswan #

But when i remove it and reload it,it looks correct:

Code: Select all

office /home/rdswan # rmmod tg3 && modprobe tg3
office /home/rdswan # lsmod|grep tg3
tg3                   184320  0
libphy                 77824  3 tg3,broadcom,bcm_phy_lib
office /home/rdswan #

and then eth0 starts right up:

Code: Select all

office /home/rdswan # /etc/init.d/net.eth0 start
 * Executing: /lib64/rc/sh/openrc-run.sh /lib64/rc/sh/openrc-run.sh /etc/init.d/net.eth0 start
 * Bringing up interface eth0
 *   Skipping module adsl due to missing program: /usr/sbin/adsl-start /usr/sbin/pppoe-start
 *   Skipping module br2684ctl due to missing program: br2684ctl
 *   Skipping module clip due to missing program: /usr/sbin/atmsigd
 *   Skipping module ethtool due to missing program: ethtool
 *   Skipping module netplugd due to missing program: /sbin/netplugd
 *   Skipping module ifplugd due to missing program: /usr/sbin/ifplugd
 *   Skipping module ipppd due to missing program: /usr/sbin/ipppd
 *   Skipping module iwconfig due to missing program: /sbin/iwconfig
 *   Skipping module firewalld due to missing program: firewall-cmd
 *   Skipping module pump due to missing program: /sbin/pump
 *   Loaded modules: apipa arping bonding l2tp tuntap bridge ccwgroup dummy hsr macvlan macchanger macnet wpa_supplicant ssidnet iproute2 pppd system vlan dhcpcd ip6rd ip6to4
 *   ip link set dev eth0 up
 *   Configuring eth0 for MAC address 5C:F9:DD:6C:CC:26 ...                                                                                                                                                                           [ ok ]
 *   dhcp ...
 *     Running dhcpcd ...
sending commands to master dhcpcd process                                                                                                                                                                                             [ ok ]
 *     received address 192.168.1.174/24                                                                                                                                                                                              [ ok ]
 *   ip -4 route flush table cache dev eth0
 *   ip -6 route flush table cache dev eth0
office /home/rdswan #

So again - all the modules work, and so does the device. But somethings up with the ordering of it all...

Jaglover · Post by **Jaglover** » Wed Jun 20, 2018 1:44 am

Can you pastebin the whole dmesg, please.

russK · Post by **russK** » Wed Jun 20, 2018 3:02 am

Whole dmesg would help yes.
I'm wondering if it's firmware related.

Regards

beerbellyswan · Post by **beerbellyswan** » Wed Jun 20, 2018 12:31 pm

I would think it's this part (which i assume is when tg3 is being added at boot):

Code: Select all

[    6.827496] igb: Copyright (c) 2007-2014 Intel Corporation.
[    6.834278] pcnet32: pcnet32.c:v1.35 21.Apr.2008 tsbogend@alpha.franken.de
[    6.845812] sky2: driver version 1.30
[    6.862417] tg3.c:v3.137 (May 11, 2014)
[    6.869818] libphy: tg3 mdio bus: probed
[    6.869820] tg3 0000:03:00.0: No PHY devices
[    6.869849] tg3 0000:03:00.0: Problem fetching invariants of chip, aborting
[    6.880776] scsi 6:0:0:0: Direct-Access     Generic- SD/MMC           1.00 PQ: 0 ANSI: 0

Later on i see better messaging (which i assume is when tg3 is removed/added manually be me):

Code: Select all

[   30.834074] caller _nv001169rm+0xe3/0x1d0 [nvidia] mapping multiple BARs
[   30.996897] nvidia-modeset: Allocated GPU:0 (GPU-c6a60c2d-f350-339c-2f3d-2d4947294efa) @ PCI:0000:01:00.0
[  225.640546] tg3.c:v3.137 (May 11, 2014)
[  225.671773] libphy: tg3 mdio bus: probed
[  225.731489] Broadcom BCM57780 300:01: attached PHY driver [Broadcom BCM57780] (mii_bus:phy_addr=300:01, irq=POLL)
[  225.731888] tg3 0000:03:00.0 eth0: Tigon3 [partno(BCM57788) rev 57780001] (PCI Express) MAC address 5c:f9:dd:6c:cc:26
[  225.731891] tg3 0000:03:00.0 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
[  225.731893] tg3 0000:03:00.0 eth0: dma_rwctrl[76180000] dma_mask[64-bit]
[  225.928849] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  225.929430] tg3 0000:03:00.0 eth0: Link is down
[  230.049090] tg3 0000:03:00.0 eth0: Link is up at 1000 Mbps, full duplex
[  230.049092] tg3 0000:03:00.0 eth0: Flow control is off for TX and off for RX
[  230.049103] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

For the complete dmesg:
https://paste.pound-python.org/show/ux7 ... iMYLrA3L3/

krinn · Post by **krinn** » Wed Jun 20, 2018 2:48 pm

Code: Select all

[    0.019269] core: PMU erratum BJ122, BV98, HSD29 workaround disabled, HT off
[    0.009501] core: PEBS disabled due to CPU errata, please upgrade microcode
[    0.035760] pci 0000:00:01.0: ASPM: current common clock configuration is broken, reconfiguring
[    0.041997] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[    0.042215] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[    0.042431] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0

This make a lot of, well, shit.
The problem with the tg3 is certainly because the bios didn't assign an irq to the card when the driver gets load (the card use either LNKE LNKF or LNKG which are invalid with an irq0 set, irq0 should be the one for the timer, and it's never good to share this one with other devices)
On next attempt, it succeed by polling irq.

What you could do:
* update your bios (but i could guess your m/b manufacturer sucks so much that i wouldn't be surprise no update exist).
* boot with irqpoll=yes set (a workaround, certainly the one that could easy solve your issue)
* you should update microcode earlier (yes sadly, initrd method), cpu is update but the PEBS error comes prior the update (i don't have any idea of what this does, but this doesn't still look good)
* try switching to hpet timer (at least 100% working for me) you can set it in kernel or boot with clocksource=hpet)
* kick your m/b manufacturer's ass, he really deserve it
* your bios should have a "pnp bios" feature, enable it if disable, disable it if enable (yeah i know, weak, but with buggy bios, that's to try)
* use your own kernel rather than genkernel (i never trust something i couldn't understand, and i have never understand genkernel)
* as an easy test, use a livecd (sysrescuecd is pretty nice), to see how its kernel handle your, well, mess.
* some bios offer acpi versions implementation, try switch this too
* i think i remember someone showing a doc on howto fix ACPI, but i cannot point you to it

who the hell has done this shitty board?

beerbellyswan · Post by **beerbellyswan** » Wed Jun 20, 2018 8:25 pm

Thanks for the detailed list of potential culprits krinn - unfortunately I dont think that's the problem. The m/b manufacturer is DELL, and the BIOS is the latest version of it (i did just check). I also tried booting in with the two options (irqpoll=yes clocksource=hpet), but neither worked. Not sure what you mean by "update microcode earlier" so i didn't try that

latest dmesg here:
https://paste.pound-python.org/show/PfW ... p6rXZoBks/

russK · Post by **russK** » Thu Jun 21, 2018 1:09 am

Someone battling the same issue 6 years ago?:

http://www.insentricity.com/a.cl/180/ge ... dependency

beerbellyswan · Post by **beerbellyswan** » Thu Jun 21, 2018 12:51 pm

@russK - yep, i tried that as well with no luck. the same exact messaging exists during boot.

tg3 kernel module not loaded at boot prior to net.eth0 setup

tg3 kernel module not loaded at boot prior to net.eth0 setup

Re: tg3 kernel module not loaded at boot prior to net.eth0 s