grub2 + lvm2-raid1 - initrd = [just too soon - solved]

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

TL;DR: Now that I understand how to do it, Neddy has pointed out that early init of lvm raid volumes really isn't ready for use yet...

Hi guys,

This is a new install and I'm trying something new.
I'm trying to avoid using an initial ramdisk.
I'm pretty sure I'm missing a kernel option... or?

GPT disks have a bios partition and a lvm-raid1 partition.
The lvm-raid1 partition has boot/swap/root volumes.

Grub launches fine and from its command line I can list and access all lvm partitions.
("grub> ls -l (lvm/raid1-root)/" lists the root file system contents.)

The problem is the kernel panics on boot with:
"not syncing: VFS: Unable to mount root fs on unknown-block(0,0)"

The system chroots and works fine from a gentoo-minimal disk.

Grub-mkconfig generated grub.cfg currently has:
"linux /vmlinuz-5.8.8 root=/dev/mapper/raid1-root ro init=/lib/systemd/systemd"

I've also tried these with the same result:
"linux /vmlinuz-5.8.8 root=lvm/raid1-root ro init=/lib/systemd/systemd"
"linux /vmlinuz-5.8.8 root=UUID=<correct root uuid> ro init=/lib/systemd/systemd"

I've read through various how-tos on grub2/lvm2/lvm-raid1 and have rebuilt the kernel several times, adding in various "recommended" kernel options. Maybe that isn't my issue? My reading suggests this should work. Any pointers would be greatly appreciated...

pietinger · Posted: Thu Sep 17, 2020 5:16 pm Post subject:

Maybe you need the kernel(-module) parameter "dm-mod.create=" (dm_init) ? With this you can set needed parameters for all dm-modules .

(an example for this - but with encrypting instead of a raid - you will find here: https://forums.gentoo.org/viewtopic-t-1110764-highlight-.html
see also "raid_tests()" in https://gitlab.collabora.com/koike/dm-cmdline-test/-/blob/master/dm-cmd-test.sh)

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

Thanks @pietinger

I do have dm_init compiled into the kernel but figured grub would do some magic and pass whatever it needed... I guess not. So I checked out your links and looked through several dm-mod LVM examples.

The kernel docs say, "The dm line should be equivalent to the one used by the dmsetup tool with the –concise argument." I added the relevant volume from "dmsetup --concise table" into /etc/defaults/grub and ended up with:

linux /vmlinuz-5.8.8 root=/dev/mapper/raid1-root ro dm-mod.create="raid1-root,<correct lv uuid>,14,rw,0 577339392 raid raid1 3 0 region_size 1024 2 253:10 253:11 253:12 253:13" init=/lib/systemd/systemd

Unfortunately there is no change in boot behavior.

Since the docs also say to use the "UUID" and the dmsetup command returns "LV UUID"s, I tried replacing the lv uuid with the uuid from blkid... Also no change in boot behavior.

No uuid at all, as shown in most examples, does not boot either.

I also tried including the entire (rather large) string returned by dmsetup thinking maybe it needed it all... Again, no go.

I did verify the changes were making it into grub.cfg before each attempt to boot.

Still poking around and trying different things...

pietinger · Posted: Thu Sep 17, 2020 8:57 pm Post subject:

I think the error is here:

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

Hmm. I thought that was the point of dm-mod.create? From the device mapper doc about Early creation of mapped devices: (https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-init.html)

NeddySeagoon · Posted: Sat Sep 19, 2020 7:46 pm Post subject:

drumgod,

The table at https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-init.html says raid is not allowed.
Its going to be very fragile too. If you grow lv-root to need to fix the kernel command line.

I was trying to do an initrd free root in lvm on top of kernel raid5 but dm-init tries to start the logical volume before the raid set is assembled.
My notes are at https://wiki.gentoo.org/wiki/User:NeddySeagoon/Root_Inside_LVM_On_Raid

raid on its own works. dm-mod.create= on its own works.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

NeddySeagoon · Posted: Sun Sep 20, 2020 10:48 am Post subject:

drumgod,

There are two sorts of raid which we may be confusing.
mdadm raid, which is entirely separate to LVM.
and LVM built in raid.

When you create a PV spread across several spindles, you can chose how the several spindles are used.
linear, is the space is allocated end to end, like one large partition.
raid means that space is allocated in some raid like way, I have not tried different levels) and lvm does the raid work, not mdadm.
Its this raid that is not permitted with dm-mod.create.

I was trying mdadm raid auto assemble with raid5 and dm-mod.create on top.
As i said, both bits work on their own but the kernel performs the dm-mod.create before the raid set is assembled, so the layers fail.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

Thanks Neddy,

NeddySeagoon · Posted: Sun Sep 20, 2020 10:59 pm Post subject:

drumgod,

I would not go so far as to say, will not work. The Target types table says.
raid constrained, userspace should verify metadata device.

That only says that horrible things might happen. :)

I read that before I put my experimental system together. Calling it raid5 is a bit of an exaggeration.
Its two partitions on the same spindle with the third partition missing :)

Lets look at the kernel code. Read the source, Luke. :)

drumgod · n00b Joined: 12 Dec 2003 Posts: 61

lol! Well that takes away any question as to whether this configuration will work. As the code stands, it will not. Thank you for patiently pointing it out.

Just a little too close to the edge I guess.

Hmmm. Lets see if I can get it to work with an initrd... I appreciate the help guys!