dm-* device & volume group not available at boot [SOLVED]

[NathanZachary]

Hello all,

I have an odd problem that I'm trying to troubleshoot. When I boot a system that has two volume groups (call them vg00 and vg01), both attached to the same hardware RAID controller, only one of the VGs shows at boot. Unfortunately, it's not the VG containing the root partition. Further, the corresponding /dev/dm-* devices for vg00 aren't showing up either. However, when I boot into a live environment, both VGs and all of the /dev/dm-* devices load properly. I have tried regenerating the initramfs as well as issuing `vgmknodes`, but haven't been able to fix the problem. It's strange to me for two reasons: 1) one of the VGs is found at boot, but the other one isn't; 2) they are both found when booting into a live environment.

Also, I should mention that when I'm dropped into the initramfs shell, the partitions show when using `blkid`. So, it seems like the kernel is able to see the underlying disks but is not able to find the details about the corresponding device mapper entries or that particular volume group.

I would appreciate any suggestions as to where to look next.

Thank you.

Cheers,
Nathan Zachary
_________________
“Truth, like infinity, is to be forever approached but never reached.” --Jean Ayres (1972)
---avatar cropped from =AimanStudio---

[zen_desu]

[NathanZachary]

It's a BusyBox initramfs. When it drops to the shell, I can issue the `vgchange` command to activate vg01, but not vg00. It says that the latter doesn't exist. That's the part that is perplexing to me is how one is showing up, but not the other. Since both volume groups are comprised of the same type of physical disks attached to the same controller, it shouldn't be a missing kernel driver/module. Further, since I can see the partitions that comprise vg00, it seems to me like it's a problem with device mapper not creating or recognising the corresponding dm-* devices upon boot.
_________________
“Truth, like infinity, is to be forever approached but never reached.” --Jean Ayres (1972)
---avatar cropped from =AimanStudio---

[grknight]

Have you tried vgscan --mknodes? What does vgs say? Also, are the physical volumes recognized with pvs? If not, force a physical rescan with pvscan --cache -aay.

[NathanZachary]

Yes, I have tried the commands that you suggested. The physical volumes tied to the OS volume (vg00) are not showing, even after a rescan with `pvscan --cache -aay`, however the physical volumes for the secondary volume (vg01) ARE showing in the BusyBox shell. Also, the partitions that make of the physical volumes in vg00 (e.g. /dev/sdb1, /dev/sdb2 and so on) are present, and the partitions are seen in `blkid`, so I don't think it's a problem with kernel modules being loaded for the disk controller.

All physical volumes, volume groups, and logical volumes are found when booting into SystemRescueCD though. As such, I suspect there is some problem with the kernel, initramfs, or device mapper on the host.
_________________
“Truth, like infinity, is to be forever approached but never reached.” --Jean Ayres (1972)
---avatar cropped from =AimanStudio---

[Hu]

Were both the good and bad PVs created using the same tool version and options? From an environment where both are visible, is there anything suspicious when comparing the output of pvs -v (or even pvs -d) for the two PVs?

[grknight]

If this is the case, I would focus on PV detection with pvscan --cache -vvvv to be super verbose and also look at dmesg. Also try pvscan --cache -vvvv <device> to make the device mapper scan that device only.

Also check dmesg while doing other such operations.

In addition, make sure all required device mapper options, such as RAID, are included in the kernel. Modules may work, but having a root not show up may indicate issues here.

[NathanZachary]

Sigh... I figured it out. There was an improper LVM filter that was causing the PVs to be excluded.
Thank you to everyone who provided suggestions.
_________________
“Truth, like infinity, is to be forever approached but never reached.” --Jean Ayres (1972)
---avatar cropped from =AimanStudio---