Mounting software raid (mdadm) array on boot

[nydas]

Hi all,

My first post following my first *successful* install of Gentoo. I tried back in 2005 but failed miserably, but am happy to have a booting system However all is not quite right and I'm hoping for some assistance. I'll try to provide as much info as possible, but happy to provide any more if it's going to help.

I'm booting off /dev/sda which is a 2TB drive. All good.

I have /dev/sd{fghi} which are each 4TB in size, and I've used mdadm to configure these as RAID6. It's listed in my fstab to mount at /home however won't mount when booted. Once booted however, if I login and 'mount -a' then it will mount without issue. I had thought that perhaps it wasn't in the initramfs, but I've tried doing a full recompile and it's still not working.

So onto the code snaps...

/etc/fstab

Code: Select all

# /etc/fstab: static file system information.

# <fs>			<mountpoint>	<type>		<opts>		<dump/pass>
#/dev/cdrom		/mnt/cdrom	auto		noauto,ro	0 0
/dev/sda1		/boot		vfat		defaults	0 2
/dev/sda2		none		swap		sw		0 0
/dev/sda3		/		ext4		noatime		0 1
UUID=0534d246-53df-4bf5-b03e-47200d9557dd		/home		ext4		noatime		0 2

It's the UUID that's the 'offending item'.

blkid /dev/md127

Code: Select all

/dev/md127: LABEL="NydasCoreRaid1" UUID="0534d246-53df-4bf5-b03e-47200d9557dd" BLOCK_SIZE="4096" TYPE="ext4"

mdadm --detail --scan

Code: Select all

ARRAY /dev/md/livecd:NydasCoreRaid1 metadata=1.2 name=livecd:NydasCoreRaid1 UUID=63fa5173:12b79cb4:e75da233:4986f28d

/etc/mdadm.conf

Code: Select all

ARRAY /dev/md/livecd:NydasCoreRaid1 metadata=1.2 name=livecd:NydasCoreRaid1 UUID=63fa5173:12b79cb4:e75da233:4986f28d

On boot I get the following:

Code: Select all

...
* Starting up RAID devices				!!
...
fsck.ext4: Unable to resolve 'UUID=0534d246-53df-4bf5-b03e-47200d9557dd'
* Operational error
...
...
mount: /home: can't find UUID=0534d246-53df-4bf5-b03e-47200d9557dd.
* Some local filesystem failed to mount
...

I've tried rebuilding the kernel and initramfs using the following:

Code: Select all

genkernel all --mdadm
grub-install /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg

However this doesn't appear to have made any difference. Like I say, once booted and logged in, I can 'mount -a' and it will mount /home without complaint.

Any guidance would be really appreciated. Having tried and failed at Gentoo back in 2005, there will be a certain level of satisfaction if I can actually get this last bit working!

Thank you in advance,

Andy

[Hu]

Since mount works later, that suggests the initial mount occurs too soon, before the MD device is ready. You elided much of the boot output, and I suspect there was something of value there. What in the initramfs do you expect to set up the MD device? In your boot output, it looks like the MD device does not set up properly. What error does it print?

[nydas]

Since mount works later, that suggests the initial mount occurs too soon, before the MD device is ready. You elided much of the boot output, and I suspect there was something of value there. What in the initramfs do you expect to set up the MD device? In your boot output, it looks like the MD device does not set up properly. What error does it print?

Hi,

I think you're right. Just not sure how to solve it.

I hadn't written out the full boot output, and couldn't find a way here to add an image. But I've hosted one here. I can't get all of it on one screen, however there is no error before what is shown, and there isn't much after the network is online - certainly no errors. I've also posted my dmesg output here.

My understanding was that passing --mdadm into the build of the initramfs was all that was required, however I'm clearly mistaken.

Does this help?

Andy

[sabayonino]

I have 2 mirror disks on my gentoo system.

Code: Select all

cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] 
md127 : active raid1 sdb1[2] sdc1[1]
      976629568 blocks super 1.2 [2/2] [UU]
      
unused devices: <none>



ARRAY /dev/md127 metadata=1.2 spares=1 name=gentoo:0 UUID=67b421e6:431ca07a:83c8f084:4ed88c95

Not sure , but try to add "domdadm" to grub option

Code: Select all

GRUB_CMDLINE_LINUX_DEFAULT="[...] domdadm"

update grub config file

or check the real UUID running "blkid"

Code: Select all

/dev/md127: UUID="fdba8b95-68fe-44d0-b377-0c6459f5e9d1" BLOCK_SIZE="4096" TYPE="ext4"

to mount <mount_point>

My FStab

Code: Select all

UUID=fdba8b95-68fe-44d0-b377-0c6459f5e9d1       /mnt/nfs     ext4    defaults,errors=remount-ro      0 2

[NeddySeagoon]

nydas,

/home need not be assembled in the initrd as its not required to start the system.
It needs to be assembled before localmount runs, as that's the service that mounts it.

Code: Select all

...
* Starting up RAID devices            !!
...
fsck.ext4: Unable to resolve 'UUID=0534d246-53df-4bf5-b03e-47200d9557dd'
* Operational error 

Why is something trying to fsck /home when its ext4?
It should be a journal replay, not a fsck.

[nydas]

Sabayonino,

My grub has the following:

Code: Select all

linux /vmlinuz-5.10.27-gentoo-x86_64 root=UUID=<xyz> ro domdadm
echo 'Loading initial ramdisk ...'
initrd /initramfs-5.10.27-gentoo-x86_64.img

<xyz> is actual UUID but I didn't want to type it all out.

FStab is the correct UUID, but using noatime rather than defaults,errors=remount-ro, but that shouldn't make a difference.

NeddySeagoon,

I'm afraid I can't answer your question on fsck. I guess fsck.ext4 exists for a reason?

[Hu]

As a wild guess regarding fsck.ext4: ext4 filesystems can be run without a journal, so perhaps the call to fsck.ext4 is standard, and most of us never notice it because in normal operation, the journal replay has already handled everything, and there is either no message or a benign status message we ignore. In this scenario, since the device cannot be found, fsck.ext4 cannot determine that it has nothing to do, so instead it outputs an error that it failed to find the device. That brings us back to, why is the device not found at that stage, but instead assembled later?

[NeddySeagoon]

nydas,

Tell us how you make your kernel and initrd.

If you make your own kernel, please put the .config file onto a pastebin site.

[nydas]

Sorry for the late reply - I've been slammed at work.

This is my first foray into Gentoo so I went with the easy option and simply did a genkernel and passed mdadm in.

I did find a *overflow.com post where someone on Arch was facing a similar issue. Their solution (assumed working by the fact it had a green tick and two upvotes) was simply reduce the information within the mdadm.conf file to /dev/md0 UUID=xyz (noting it not the same UUID as provided by blkid). I tried this, however unfortunately it didn't help. My device is however now linked to md0 instead of md127, and can still be manually mounted after login. I'm taking this as a small success.

Andy

[NeddySeagoon]

nydas,

mdadm takes minor device number from the config if its given, if not, it starts at 127 and works its way down.

Code: Select all

mdadm -E /dev/<underlying device>

tells you all about the raid set that <underlying device> is part of, incluhing the UUID of the the set that you pass to the assemble command.

[Whissi]

Are you really booting genkernel's initramfs? If you only use genkernel for initramfs generation, it is crucial to pass --kernel-config to genkernel so that the generated initramfs will work with your kernel config.

When you boot genkernel initramfs, make sure you pass domdadm as kernel argument like told by genkernel or genkernel will not invoke mdadm.

[nydas]

So I found a resolution, in a round-about kinda way.

I've added _netdev into my fstab for that mount, which pushes it down the boot sequence. By the time it has connected my network, it's able to mount the device.

So while it's not perfect (as in this is a bit of a hack), it seems to work.

Not sure if this counts as a 'closed thread', but if others get stuck this seems to do the trick.

[NeddySeagoon]

nydas,

I would suggest that its a bug and should be reported at bugs.gentoo.org.

[LiuYY]

Add the service "udev-settle" can solve this problem.

Code: Select all

rc-update add udev-settle sysinit

Refer to the last chapter of :
https://wiki.gentoo.org/wiki/Udev