Config_macsec

[davidbryant]

I am slowly working through the long list of device drivers, getting rid of the ones I do not need. I ran into an odd situation with "CONFIG_MACSEC". Unless I set this option to either "Y" or "M", my system won't boot. I get this message:

Code: Select all

Block device UUID=95358233-7da4-47b7-9f5a-72d0da20830d is not a valid root device

Then the system halts, and prompts me for a root device.

Clearly I have to set CONFIG_MACSEC = Y (or M). Here's what puzzles me. This flag is part of the section Device Drivers --> Network Device Support. The description of that section says

Code: Select all

Network device support (NETDEVICES)

CONFIG_NETDEVICES:

You can say N here if you don't intend to connect your Linux box to
any other computer at all.

You'll have to say Y if your computer contains a network card that
you want to use under Linux. If you are going to run SLIP or PPP over
telephone line or null modem cable you need say Y here. Connecting
two machines with parallel ports using PLIP needs this, as well as
AX.25/KISS for sending Internet traffic over amateur radio links.
Type : tristate

See also "The Linux Network Administrator's Guide" by Olaf Kirch and
Terry Dawson. Available at <http://www.tldp.org/guides.html>.

If unsure, say Y.

Symbol: NETDEVICES [=y]
Type : bool
Defined at drivers/net/Kconfig:6
Prompt: Network device support
Depends on: NET [=y]
Location:
-> Device Drivers
Selected by [m]:
- SCSI_CXGB3_ISCSI [=m] && SCSI_LOWLEVEL [=y] && SCSI [=y] && PCI [=y] && INET [=y] && (IPV6 [=n] || IPV6 [=n]=n)
- SCSI_BNX2_ISCSI [=m] && SCSI_LOWLEVEL [=y] && SCSI [=y] && NET [=y] && PCI [=y] && (IPV6 [=n] || IPV6 [=n]=n) && MMU [=y]
- SCSI_BNX2X_FCOE [=m] && SCSI_LOWLEVEL [=y] && SCSI [=y] && PCI [=y] && (IPV6 [=n] || IPV6 [=n]=n) && LIBFC [=m] && LIBFCOE [=m] && MMU [=y]

The description of MACSEC (one of ~30 options under Network Device Support) is fairly innocuous:

Code: Select all

IEEE 802.1AE MAC-level encryption (MACsec) (MACSEC)

CONFIG_MACSEC:

MACsec is an encryption standard for Ethernet.

Symbol: MACSEC [=y]
Type : tristate
Defined at drivers/net/Kconfig:294
Prompt: IEEE 802.1AE MAC-level encryption (MACsec)
Depends on: NETDEVICES [=y] && NET_CORE [=y]
Location:
-> Device Drivers
-> Network device support (NETDEVICES [=y])
-> Network core driver support (NET_CORE [=y])
Selects: CRYPTO [=y] && CRYPTO_AES [=y] && CRYPTO_GCM [=y] && GRO_CELLS [=y]

My root device is a SCSI disk. So there's a clue (sort of) that I might need MACSEC = Y to boot the system. But the Linux documentation embdded in "xconfig" really doesn't make that clear. Any ideas how I can get that documentation improved? It took a long time to isolate this problem, and I'd like to help prevent others from repeating my trial and error debugging process. Thanks.

[davidbryant]

A second item in the section Device Drivers --> Network Device Support causes the kernel to freeze on booting. (not a valid root device). That flag is named CONFIG_NETCONSOLE. The configuration hint says "If you want to log kernel messages over the network, enable this. See <file:Documentation/networking/netconsole.rst> for details." I read that fle all the way through, and concluded I didn't need "netconsole", which is described as an ethernet interface for posting kernel log messages. I was wrong. For some reason, Linux can't recognize my root device unless this flag is set. to "Y" or "M". Go figure.

[Hu]

You have a very strange setup. I've never needed either of those enabled on any of the systems for which I built kernels. How is your root device attached to the system? What kind of disk is it? What kernel driver actually operates it? How are you generating your initramfs in the good and bad cases?

[davidbryant]

You have a very strange setup. I've never needed either of those enabled on any of the systems for which I built kernels. How is your root device attached to the system? What kind of disk is it? What kernel driver actually operates it? How are you generating your initramfs in the good and bad cases?

Maybe it is strange. I don't thinlk so. Stock PC that I bought straight from the manufacturer in Round Rock, TX. I used to tinker with my PCs by adding extra components. etc., but I've never even cracked the case on this one. Here are the answers to your four questions.

1. SCSI disk in a Dell XPS8930 box. I think it's probably an SATA cable connecting the HDD to the motherboard, but I've never looked.

2. Seagate "Barracuda" (ST1000M010-2EP102 - 931.51MB) I got that information from gparted. I don't have a complex disk setup; just a /home partition, a swap partition, a /boot/efi partition, and a / partition (everything else).

3. I'm not certain that there is a kernel "driver". SCSI support is built into the kernel. I set CONFIG_BLK_DEV_SD = Y. That option says

Code: Select all

If you want to use SCSI hard disks, Fibre Channel disks, Serial ATA (SATA) or Parallel ATA (PATA) hard disks, USB storage or the SCSI or parallel port version of
the IOMEGA ZIP drive, say Y and read the SCSI-HOWTO, the Disk-HOWTO and the Multi-Disk-HOWTO, available from <http://www.tldp.org/docs.html#howto>. This is 
NOT for SCSI CD-ROMs.

To compile this driver as a module, choose M here and read <file:Documentation/scsi/scsi.rst>. The module will be called sd_mod.

Do not compile this driver as a module if your root file system (the one containing the directory /) is located on a SCSI disk. In this case, do not compile the 
driver for your SCSI  host adapter (below) as a module either.

4. I haven't been regenerating the initramfs file; just tweaking the .config file and re-compiling the kernel. Sometimes I also reinstall the modules (when some of them have been re-c
I haven't worried about the "SCSI-HOWTO" stuff before today. I guess I'll go read that in a little while. I have always set CONFIG_BLK_DEV_SD = Y, ever since I started configuring my own kernel (about wo years ago). Most of the time it just works.

4. I haven't been regenerating the initramfs file; just tweaking the .4. I haven't been regenerating the initramfs file; just tweaking the .config file and re-compiling the kernel. Sometimes I also reinstall the modules (when some of them have beeconfig file and re-compiling the kernel. Sometimes I also reinstall the modules (when some of them have been re-compiled). I had been using dracut to build the initramfs, but that stopped working correctly about three days ago, so I switched back to "genkernel ramdisk". The current initramfs was generated this way:

Code: Select all

genkernel ramdisk --kernel-config=/usr/src/linux/.config

Thanks!

[Hu]

There are some odd text bits in your response. It looks like copy/paste editing got away from you. I think I see what you were trying to say though.

A stock PC with root on a locally attached SCSI/SATA setup is very simple. I was expecting something much more exotic. Locally attached SCSI should be accessible even with NET=n. No wireless or cryptography should be necessary. Using UUID= requires an initramfs to find the filesystem with the stated UUID. If your initramfs were broken, or had broken tools in it, that could explain the boot failure. I don't see any hits for is not a valid root in the kernel source, so I assume that message must come from your initramfs. I suppose it's possible that the SCSI disk is not ready quickly, and adding MACSEC slows down the startup enough that the disk is ready before the initramfs searches for it. However, I would expect that a locally attached disk ought to be ready very quickly. I'm not familiar with dracut or genkernel, so I cannot advise how to debug them specifically.

When the system prompts you for a root device, what do you enter to get it working? Does the boot work with MACSEC=n if you set root= on the kernel command line to the same value that you enter at this prompt? Do you see any kernel messages to suggest that sda was not ready when the initramfs tried to resolve the UUID? Does this work if you use a PARTUUID instead of a UUID? PARTUUID can be resolved without an initramfs; UUID cannot. (Though, if you have an initramfs, I think it is responsible for handling any root=.)

[pa4wdh]

It seems odd to me that a network driver would cause the root device to be found (or not), which makes me think it might be a timing issue:
The root disk might take some time to become available, when you add more drivers booting takes a fraction of a second longer, which might be just enough the the disk to work.
If this is really the case you could try to add the rootwait parameter to the kernel which instructs it to wait for the root device to become available.

[davidbryant]

It seems odd to me that a network driver would cause the root device to be found (or not),... add the rootwait parameter to the kernel which instructs it to wait for the root device to become available.

Yeah, it seems odd to me, as well. That's why I asked a question in the first place. I tried adding "rootwait" to the kernel command line. That didn't help. I read some other forum posts on the same subject. They suggested adding "rootfstype=ext4" to the kernel command line. That doesn't help, either.

I can't actually show you what the system said just before it froze up, because it dies before sysklogd (my logging utlity) has been started. But I did verify that everything looks pretty normal, and that these are the messages displayed just before the error message "Block device UUID=... is not a valid root device" is issued.

Code: Select all

Feb 14 08:07:25 localhost kernel: ima: Allocated hash algorithm: sha256
Feb 14 08:07:25 localhost kernel: ima: No architecture policies found
Feb 14 08:07:25 localhost kernel: printk: console [netcon0] enabled
Feb 14 08:07:25 localhost kernel: netconsole: network logging started
Feb 14 08:07:25 localhost kernel: RAS: Correctable Errors collector initialized.
Feb 14 08:07:25 localhost kernel: Freeing unused kernel image (initmem) memory: 2844K
Feb 14 08:07:25 localhost kernel: Write protecting the kernel read-only data: 24576k
Feb 14 08:07:25 localhost kernel: Freeing unused kernel image (text/rodata gap) memory: 2036K
Feb 14 08:07:25 localhost kernel: Freeing unused kernel image (rodata/data gap) memory: 40K
Feb 14 08:07:25 localhost kernel: x86/mm: Checked W+X mappings: passed, no W+X pages found.
Feb 14 08:07:25 localhost kernel: rodata_test: all tests were successful
Feb 14 08:07:25 localhost kernel: Run /init as init process

I suppose the error message is triggered when Linux tries to mount my / directory. At least, that seems logical to me. "Init" is going to need the directory tree to find all the stuff it's supposed to start running.

[davidbryant]

Just on a hunch, I re-generated my initramfs file. That fixed the problem. Apparentlly there are some interdependencies among the various modules in the initramfs archive. I'll try to track the exact difference down later today. For now I suppose I ought to recreate the initramfs whenever I update the kernel. Most of the time my old initramfs still works. But not always.

[Hu]

Code: Select all

Feb 14 08:07:25 localhost kernel: Run /init as init process

I suppose the error message is triggered when Linux tries to mount my / directory. At least, that seems logical to me. "Init" is going to need the directory tree to find all the stuff it's supposed to start running.

The kernel handed off control to the /init program in your initramfs. That /init should have mounted your hard drive's root directory, and transferred control to /sbin/init from the hard drive. Instead, it failed with the error about the UUID= not found. More output would be helpful, but since you solved it, my guess is that the initramfs didn't scan the correct device node when searching for filesystems with the given UUID. That could be because the device was not ready in time. It could be because the kernel had not yet loaded a necessary module. (Though for a simple directly attached SCSI disk, that explanation seems unlikely.)

Personally, I make my kernels non-modular, and the initramfs never needs to be regenerated, because it is independent of the kernel version used.

[davidbryant]

Personally, I make my kernels non-modular, and the initramfs never needs to be regenerated, because it is independent of the kernel version used.

Well, I'm working in that direction. I've got the number of modules whittled down quite a bit from the place I started at. But there are an awful lot of them. ~800 still to go.

Thanks for the explanation.