Gentoo Forums :: View topic - Unable to fully initialize a Chelsio T420-BT PCIe adapter

Unable to fully initialize a Chelsio T420-BT PCIe adapter

View unanswered posts
View posts from last 24 hours

Gentoo Forums Forum Index

Networking & Security

View previous topic :: View next topic

Author

Message

blinky
n00b
n00b

Joined: 14 Sep 2016
Posts: 5

Posted: Mon Sep 26, 2016 10:07 am Post subject: Unable to fully initialize a Chelsio T420-BT PCIe adapter

First off I'm quite new to Gentoo.

I have a Chelsio T420-BT that doesn't seem to want to initialize. The kernel reports that the physical/virtual functions (SR-IOV) appear to initialize but the physical NIC itself doesn't. There were some kernel warnings that I don't understand:

Kernel log:
full kernel log here
0000:01:00.4 is the physical NIC in question.
0000:01:00.0 -> 3 are the 'physical functions'

Code:

[ +0.000091] pci 0000:01:00.0: [1425:4009] type 00 class 0x020000
[ +0.000019] pci 0000:01:00.0: reg 0x10: [mem 0xf7380000-0xf73bffff 64bit]
[ +0.000016] pci 0000:01:00.0: reg 0x20: [mem 0xf73cc000-0xf73cdfff 64bit]
[ +0.000006] pci 0000:01:00.0: reg 0x30: [mem 0xf7180000-0xf71fffff pref]
[ +0.000043] pci 0000:01:00.0: reg 0x1e8: [mem 0x00000000-0x00000fff 64bit]
[ +0.000002] pci 0000:01:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0000ffff 64bit] (contains BAR0 for 16 VFs)
[ +0.000138] pci 0000:01:00.0: reg 0x1f8: [mem 0x00000000-0x00001fff 64bit]
[ +0.000002] pci 0000:01:00.0: VF(n) BAR4 space: [mem 0x00000000-0x0001ffff 64bit] (contains BAR4 for 16 VFs)
[ +0.000159] pci 0000:01:00.0: System wakeup disabled by ACPI
[ +0.000116] pci 0000:01:00.1: [1425:4009] type 00 class 0x020000
[ +0.000019] pci 0000:01:00.1: reg 0x10: [mem 0xf7340000-0xf737ffff 64bit]
[ +0.000016] pci 0000:01:00.1: reg 0x20: [mem 0xf73ca000-0xf73cbfff 64bit]
[ +0.000006] pci 0000:01:00.1: reg 0x30: [mem 0xf7100000-0xf717ffff pref]
[ +0.000038] pci 0000:01:00.1: reg 0x168: [mem 0x00000000-0x00000fff 64bit]
[ +0.000002] pci 0000:01:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0000ffff 64bit] (contains BAR0 for 16 VFs)
[ +0.000141] pci 0000:01:00.1: reg 0x178: [mem 0x00000000-0x00001fff 64bit]
[ +0.000001] pci 0000:01:00.1: VF(n) BAR4 space: [mem 0x00000000-0x0001ffff 64bit] (contains BAR4 for 16 VFs)
[ +0.000185] pci 0000:01:00.2: [1425:4009] type 00 class 0x020000
[ +0.000018] pci 0000:01:00.2: reg 0x10: [mem 0xf7300000-0xf733ffff 64bit]
[ +0.000017] pci 0000:01:00.2: reg 0x20: [mem 0xf73c8000-0xf73c9fff 64bit]
[ +0.000043] pci 0000:01:00.2: reg 0x168: [mem 0x00000000-0x00000fff 64bit]
[ +0.000001] pci 0000:01:00.2: VF(n) BAR0 space: [mem 0x00000000-0x0000ffff 64bit] (contains BAR0 for 16 VFs)
[ +0.000140] pci 0000:01:00.2: reg 0x178: [mem 0x00000000-0x00001fff 64bit]
[ +0.000001] pci 0000:01:00.2: VF(n) BAR4 space: [mem 0x00000000-0x0001ffff 64bit] (contains BAR4 for 16 VFs)
[ +0.000174] pci 0000:01:00.3: [1425:4009] type 00 class 0x020000
[ +0.000018] pci 0000:01:00.3: reg 0x10: [mem 0xf72c0000-0xf72fffff 64bit]
[ +0.000017] pci 0000:01:00.3: reg 0x20: [mem 0xf73c6000-0xf73c7fff 64bit]
[ +0.000042] pci 0000:01:00.3: reg 0x168: [mem 0x00000000-0x00000fff 64bit]
[ +0.000001] pci 0000:01:00.3: VF(n) BAR0 space: [mem 0x00000000-0x0000ffff 64bit] (contains BAR0 for 16 VFs)
[ +0.000139] pci 0000:01:00.3: reg 0x178: [mem 0x00000000-0x00001fff 64bit]
[ +0.000002] pci 0000:01:00.3: VF(n) BAR4 space: [mem 0x00000000-0x0001ffff 64bit] (contains BAR4 for 16 VFs)
[ +0.000173] pci 0000:01:00.4: [1425:4409] type 00 class 0x020000
[ +0.000018] pci 0000:01:00.4: reg 0x10: [mem 0xf7280000-0xf72bffff 64bit]
[ +0.000008] pci 0000:01:00.4: reg 0x18: [mem 0xf6800000-0xf6ffffff 64bit]
[ +0.000008] pci 0000:01:00.4: reg 0x20: [mem 0xf73c4000-0xf73c5fff 64bit]
[ +0.000062] pci 0000:01:00.5: [1425:4509] type 00 class 0x010000
[ +0.000018] pci 0000:01:00.5: reg 0x10: [mem 0xf7240000-0xf727ffff 64bit]
[ +0.000017] pci 0000:01:00.5: reg 0x20: [mem 0xf73c2000-0xf73c3fff 64bit]
[ +0.000005] pci 0000:01:00.5: reg 0x30: [mem 0xf7080000-0xf70fffff pref]
[ +0.000062] pci 0000:01:00.6: [1425:4609] type 00 class 0x0c0400
[ +0.000018] pci 0000:01:00.6: reg 0x10: [mem 0xf7200000-0xf723ffff 64bit]
[ +0.000017] pci 0000:01:00.6: reg 0x20: [mem 0xf73c0000-0xf73c1fff 64bit]
[ +0.000005] pci 0000:01:00.6: reg 0x30: [mem 0xf7000000-0xf707ffff pref]
[ +0.000058] pci 0000:01:00.7: [1425:0000] type 00 class 0x020000
[ +0.000132] pci 0000:01:00.0: BAR 11: assigned [mem 0xf73ce000-0xf73edfff 64bit]
[ +0.000127] pci 0000:01:00.1: BAR 11: no space for [mem size 0x00020000 64bit]
[ +0.000122] pci 0000:01:00.1: BAR 11: failed to assign [mem size 0x00020000 64bit]
[ +0.000119] pci 0000:01:00.2: BAR 11: no space for [mem size 0x00020000 64bit]
[ +0.000912] pci 0000:01:00.2: BAR 11: failed to assign [mem size 0x00020000 64bit]
[ +0.000113] pci 0000:01:00.3: BAR 11: no space for [mem size 0x00020000 64bit]
[ +0.000116] pci 0000:01:00.3: BAR 11: failed to assign [mem size 0x00020000 64bit]
[ +0.000131] pci 0000:01:00.0: BAR 7: assigned [mem 0xf73ee000-0xf73fdfff 64bit]
[ +0.000120] pci 0000:01:00.1: BAR 7: no space for [mem size 0x00010000 64bit]
[ +0.000091] pci 0000:01:00.1: BAR 7: failed to assign [mem size 0x00010000 64bit]
[ +0.000118] pci 0000:01:00.2: BAR 7: no space for [mem size 0x00010000 64bit]
[ +0.000095] pci 0000:01:00.2: BAR 7: failed to assign [mem size 0x00010000 64bit]
[ +0.000119] pci 0000:01:00.3: BAR 7: no space for [mem size 0x00010000 64bit]
[ +0.000082] pci 0000:01:00.3: BAR 7: failed to assign [mem size 0x00010000 64bit]
[ +0.000092] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ +0.000084] pci 0000:01:00.1: Signaling PME through PCIe PME interrupt
[ +0.000085] pci 0000:01:00.2: Signaling PME through PCIe PME interrupt
[ +0.000104] pci 0000:01:00.3: Signaling PME through PCIe PME interrupt
[ +0.000084] pci 0000:01:00.4: Signaling PME through PCIe PME interrupt
[ +0.000085] pci 0000:01:00.5: Signaling PME through PCIe PME interrupt
[ +0.000083] pci 0000:01:00.6: Signaling PME through PCIe PME interrupt
[ +0.000094] pci 0000:01:00.7: Signaling PME through PCIe PME interrupt
[ +7.602379] cxgb4 0000:01:00.4: mbox 4: 1c0002900000004 1400100001000000 0 0 0 0 0 0
[ +0.000004] cxgb4 0000:01:00.4: command 0x1 in mailbox 4 timed out
[ +0.000002] cxgb4 0000:01:00.4: Firmware reports adapter error: Crash
[ +0.000140] cxgb4: probe of 0000:01:00.4 failed with error -16

lspci:
lspci -vvnn here
01:00:7 is unassigned as stated in the user's guide.

Code:

01:00.0 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:0000]
Kernel driver in use: cxgb4
Kernel modules: cxgb4
01:00.1 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:0000]
Kernel driver in use: cxgb4
Kernel modules: cxgb4
01:00.2 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:0000]
Kernel driver in use: cxgb4
Kernel modules: cxgb4
01:00.3 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:0000]
Kernel driver in use: cxgb4
Kernel modules: cxgb4
01:00.4 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4409]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:0000]
Kernel modules: cxgb4
01:00.5 SCSI storage controller [0100]: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:4509]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:0000]
01:00.6 Fibre Channel [0c04]: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:4609]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:0000]
01:00.7 Ethernet controller [0200]: Chelsio Communications Inc Device [1425:0000]
Subsystem: Chelsio Communications Inc Device [1425:0000]

Kernel config:
full kernel config here

Code:

CONFIG_NET_VENDOR_CHELSIO=y
# CONFIG_CHELSIO_T1 is not set
# CONFIG_CHELSIO_T3 is not set
CONFIG_CHELSIO_T4=m
CONFIG_CHELSIO_T4VF=m

Firmware versions:
Sorry, I didn't know a more effective method for getting firmware versions. This is a listing of /lib64/firmware/cxgb4/ and are installed by linux-firmware. It appears to be v1.14.4

Code:

aq1202_fw.cld
bcm8483.bin
t4fw-1.14.4.0.bin
t4fw.bin
t5fw-1.14.4.0.bin
t5fw.bin

Kernel version:

Code:

4.4.8-hardened-r1 x86_64 GNU/Linux

_________________
JEEPEEJEE--BEGIN_BLOCK_McDATA
aHR0cDovL2tsaXBkLmNvbS9zY3JlZW5zaG90cy82MDIzZjczM2YyZDBhOTI0ZmVhNThjMTZkN2I4
ZDA3YS0yLmpwZwo=
---END---

NeddySeagoon
Administrator

Joined: 05 Jul 2003
Posts: 54244
Location: 56N 3W

Posted: Mon Sep 26, 2016 10:54 am Post subject:

blinky,

Welcome to Gentoo.

Notice that

Code:

01:00.5 SCSI storage controller [0100]: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:4509]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:0000]
01:00.6 Fibre Channel [0c04]: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:4609]
Subsystem: Chelsio Communications Inc T420-BT Unified Wire Storage Controller [1425:0000]

there are no drivers loaded for those elements of your card, so they won't be working just now.

Grepping the kernel source for your vendor ID

Code:

/usr/src/linux $ grep -R 1425 ./

produces one interesting hit is the SCSI region of the code.

Code:

./drivers/scsi/csiostor/csio_hw.h:#define CSIO_VENDOR_ID 0x1425

That's a SCSI driver that matches your vendor ID.

That file header says

Code:

/*
* This file is part of the Chelsio FCoE driver for Linux.

which looks promising too.
Reading the Kconfig file in that directory shows.

Code:

config SCSI_CHELSIO_FCOE
tristate "Chelsio Communications FCoE support"
depends on PCI && SCSI
depends on SCSI_FC_ATTRS

Looking in your kernel .config shows that the symbol SCSI_CHELSIO_FCOE does not appear, so you don't have that driver built.
However, we know it depends on SCSI_FC_ATTRS

Code:

# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=y
# CONFIG_SCSI_FC_ATTRS is not set

which is off, so the option is hidden.

Rebuild your kernel, using menuconfig to enable CONFIG_SCSI_FC_ATTRS which is a menu, then inside that menu, enable SCSI_CHELSIO_FCOE.
That should fix the Fibre Channel Over Ethernet part of your card.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

blinky
n00b
n00b

Joined: 14 Sep 2016
Posts: 5

Posted: Tue Sep 27, 2016 9:15 am Post subject:

Hi, thanks for taking the time to show the steps you took - invaluable information.

I deliberately didn't compile the modules for those device because I didn't think I needed them. The NIC on the card is a standard ethernet RJ45 (which is how I intend to use it) and it is function #4 - the device having the issue:

full (pci bus 01) kernel log here

Code:

[ +0.000169] pci 0000:01:00.4: [1425:4409] type 00 class 0x020000
[ +0.000019] pci 0000:01:00.4: reg 0x10: [mem 0xf7280000-0xf72bffff 64bit]
[ +0.000007] pci 0000:01:00.4: reg 0x18: [mem 0xf6800000-0xf6ffffff 64bit]
[ +0.000008] pci 0000:01:00.4: reg 0x20: [mem 0xf73c4000-0xf73c5fff 64bit]
[ +0.000080] pci 0000:01:00.4: Signaling PME through PCIe PME interrupt
[ +7.664926] cxgb4 0000:01:00.4: mbox 4: 1c0002900000004 1400100001000000 0 0 0 0 0 0
[ +0.000003] cxgb4 0000:01:00.4: command 0x1 in mailbox 4 timed out
[ +0.000002] cxgb4 0000:01:00.4: Firmware reports adapter error: Crash
[ +0.000144] cxgb4: probe of 0000:01:00.4 failed with error -16

Pardon my ignorance, and I hope you can correct me here, but would the improper initialization of other functions cause the bad behaviour seen here? More succinctly: is it necessary for all the other functions to be properly working in order for any of them to work? Function #6 is the FCOE offload device which I have no intention of using.

However, I compiled in support for it:

Code:

Although I was unable to get the iSCSI offload function working (I won't be using it): again, do you think I need this working for the whole card to function properly?

full kernel config here

Code:

CONFIG_SCSI_FC_ATTRS=y
# CONFIG_SCSI_CXGB3_ISCSI is not set
CONFIG_SCSI_CXGB4_ISCSI=m
CONFIG_SCSI_CHELSIO_FCOE=m
CONFIG_NET_VENDOR_CHELSIO=y
# CONFIG_CHELSIO_T1 is not set
# CONFIG_CHELSIO_T3 is not set
CONFIG_CHELSIO_T4=m
CONFIG_CHELSIO_T4_DCB=y
CONFIG_CHELSIO_T4VF=m

_________________
JEEPEEJEE--BEGIN_BLOCK_McDATA
aHR0cDovL2tsaXBkLmNvbS9zY3JlZW5zaG90cy82MDIzZjczM2YyZDBhOTI0ZmVhNThjMTZkN2I4
ZDA3YS0yLmpwZwo=
---END---

NeddySeagoon
Administrator

Joined: 05 Jul 2003
Posts: 54244
Location: 56N 3W

Posted: Tue Sep 27, 2016 10:12 am Post subject:

blinky,

From your first post,

Code:

01:00.0 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
01:00.1 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
01:00.2 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
01:00.3 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4009]
01:00.4 Ethernet controller [0200]: Chelsio Communications Inc T420-BT Unified Wire Ethernet Controller [1425:4409]

Notice that function 4 has a different deviceID, 4409, compared to functions 0..3, 4009, which says that in some way or another, its a different device.

The Chelsio website shows 1.15.37.0 as the latest firmware.

Code:

$ ls -l /lib/firmware/cxgb4
total 2608
-rw-r--r-- 1 root root 179202 Jul 31 12:02 aq1202_fw.cld
-rw-r--r-- 1 root root 208900 Jul 31 12:02 bcm8483.bin
-rw-r--r-- 1 root root 539648 Jul 31 12:02 t4fw-1.14.4.0.bin
-rw-r--r-- 1 root root 541184 Jul 31 12:02 t4fw-1.15.37.0.bin
lrwxrwxrwx 1 root root 18 Jul 31 12:02 t4fw.bin -> t4fw-1.15.37.0.bin
-rw-r--r-- 1 root root 593920 Jul 31 12:02 t5fw-1.14.4.0.bin
-rw-r--r-- 1 root root 596992 Jul 31 12:02 t5fw-1.15.37.0.bin
lrwxrwxrwx 1 root root 18 Jul 31 12:02 t5fw.bin -> t5fw-1.15.37.0.bin

which is what the testing version of linux-firmware gives me.
Its worth trying a firmware update.

I would not expect to load drivers for functions I wasn't going to use but there is a lot of poorly designed hardware out there, so I wouldn't rule it out either.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

blinky
n00b
n00b

Joined: 14 Sep 2016
Posts: 5

Posted: Sun Oct 09, 2016 8:13 am Post subject:

As it turns out my device is faulty - I'm ordering a replacement; I'll keep you updated. _________________ JEEPEEJEE--BEGIN_BLOCK_McDATA aHR0cDovL2tsaXBkLmNvbS9zY3JlZW5zaG90cy82MDIzZjczM2YyZDBhOTI0ZmVhNThjMTZkN2I4 ZDA3YS0yLmpwZwo= ---END---

Display posts from previous:

	Gentoo Forums Forum Index Networking & Security	All times are GMT
Page 1 of 1

Jump to:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Copyright 2001-2024 Gentoo Foundation, Inc. Designed by Kyle Manna © 2003; Style derived from original subSilver theme. | Hosting by Gossamer Threads Inc. © | Powered by phpBB 2.0.23-gentoo-p11 © 2001, 2002 phpBB Group
Privacy Policy