LVM dm-RAID(5) experiences?

eccerr0r · Posted: Mon Jan 03, 2022 1:50 am Post subject: LVM dm-RAID(5) experiences?

Curious if anyone had experience with LVM's RAID5 - meaning using MDRAID5 on top of LVM, LVM being lower on the stack closer to the disks.

I've been using LVM on top of MDRAID all these years because this was the first supported method, and it seems fairly robust. Now there's an option of using md on top of LVM.

Anyway, I tried building an LVM RAID5 in the bit of slack space I have on my RAID disks. Things went okay. Just need to remember that the number of extents you pass it is the number of blocks your actual filesystem is going to use (not counting parity blocks) and the -i option is the number of data storage disks (not counting parity disks).

What I'm not sure about is what happens when you actually get a disk failure... I thought that LVM was pretty nasty at assembly if you lose a disk, maybe not? If you are missing a PV in a VG, what does LVM do? Supposedly you still should be able to bring up the VG with missing PVs... and able to mount LV's inside that VG with missing PVs.

So what would be the equivalent of failing and kicking extents on a disk used in RAID5 over LVM? readding?
If you add a new PV to a VG with a RAID5 on it missing a volume, will it autocreate readd the missing member?

As an experiment I have a small raid5 over lvm with: /dev/sdb3 /dev/sdc3 /dev/sdd3 as PVs, created:

eccerr0r · Posted: Thu Jun 30, 2022 3:47 pm Post subject:

Had a panic disk failure last night, but it seemed to be transient. A hot unplug and replug the disk appears to be working again after verifying the disk as functional in a different machine. Was able to quickly do the dance with mdadm as I've done this many times in the past, and thus cleaned up the mdraid5 and mdraid1.

But I was at a complete loss at how to deal with the lvm RAID5 that I set up on spare partitions on the disk as an experiment.

After dropping and reattaching the disk, the LVMRAID5 still thought the old disk alive as apparently it doesn't touch the disks very often to ensure it's working even if idle. The LVMRAID5 was mounted, so I just tried writing a scratch file to the volume, deleted the file, and synced. Now it noticed the disk was dead and dropped the partition from the RAID.

So what do I do now?

The key is that one PV of the LV croaked.

Zucca · Posted: Thu Jun 30, 2022 5:45 pm Post subject:

I use the built-in raid of lvm. Which in turn uses mdraid internally. So far no problems. I just assume lvm uses mdraid wisely.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

eccerr0r · Posted: Thu Jun 30, 2022 6:03 pm Post subject:

LVM RAID (>1) ?

I see a lot of recovery information about MDRAID and LVMRAID1 but nothing more than this...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

Zucca · Posted: Fri Jul 01, 2022 7:33 pm Post subject:

I had several lvs configured as raid5 lv and raid6 lv.
There's always lvconvert --repair <lv>. But that's probably not what you're looking for.
Maybe see https://wiki.gentoo.org/wiki/LVM#Striping_with_parity_.28RAID4_and_RAID5.29 ?

If the whole raid stack gets corrupted, then... retrieve data from backups? ;)

Now that I think of lvmraid, it must be mdraid on top of lvm.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

eccerr0r · Posted: Fri Jul 01, 2022 11:39 pm Post subject:

Hmm... I see... I guess that's the "downtime" method if the RAID was disassembled and you had to reassemble it in degraded mode. I guess in my case was a hot swap.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

eccerr0r · Posted: Fri Sep 09, 2022 3:40 am Post subject:

Ugh, my disk got kicked again.

However this time it was weird, it seemed to have "fixed itself" this time? hmm... confusing. And that's why I'd rather use mdadm ...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

pjp · Administrator Joined: 16 Apr 2002 Posts: 20067

With no (useful) logs? Do you run smartctl checks on it? Maybe there was an error that was fixed.
_________________
Quis separabit? Quo animo?

eccerr0r · Posted: Fri Sep 09, 2022 3:53 pm Post subject:

Well the disk suddenly disappeared from the device table and mdraid stopped the disk from the array. However lvm-raid, as I wasn't using it at the time, I don't know.

I hot pulled the disk from the system, cleaned the contacts off real quick, and reinserted it. The disk got recognized and I was able to re-add the MDRAID partitions back to the MDRAID just as I've always done. After waiting for it to resync, I remembered I needed to deal with lvm-raid which was on the same disks in the slack space. However when I checked the lvm-raid with vgdisplay on them...

it looked healthy! All three disks were up as far as I could tell, no signs of degraded operation. Logs didn't make any mention of the lvm RAID volume either...

Weird.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

energyman76b · Advocate Joined: 26 Mar 2003 Posts: 2048 Location: Germany

I just went zfs. It is so easy and robust, mdraid looks silly in comparism.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.