GRUB/DMRAID/LVM woe

IncredibleMrT · Posted: Mon Aug 06, 2012 12:07 pm Post subject: GRUB/DMRAID/LVM woe

Hi all.
I'm trying to install gentoo on a rather old HP Proliant DL320G2 rack server. It's not the most performant of beasts but it will do just nicely for what I have planned.

It's a faily low spec Pentium4 and has two 80GB harddrives within and is 'raided' at BIOS level to raid1. I think it's known as fake raid.
The raid array has been built and initialised within the BIOS.

I start the livecd environment using

erik258 · Posted: Tue Aug 07, 2012 12:53 am Post subject:

Firstly, your post is reminiscent of:

IncredibleMrT · Posted: Tue Aug 07, 2012 8:35 am Post subject:

@erik258
The UDEV shouldn't be an issue as i'm using the standard x86 tree which is only at version 171-r6

cach0rr0 · Posted: Wed Aug 08, 2012 8:42 am Post subject:

shot in the dark -

can you confirm the kernel that's being built is including devtmpfs and devtmpfs_mount ?
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash

IncredibleMrT · Posted: Wed Aug 08, 2012 9:44 am Post subject:

Ooh @cach0rr0, you might have something there!
Just checked and it isn't included!

Rebuilding now *fingers crossed*

Just so i'm aware, as i'm using genkernel initially just to get things up and running (i'll cut it down later), do I have to specify additional settings each time I build it or should they be stored and used in the config from the previous build?

The only reason I ask is that i'm 99.9% sure i've added them before!

IncredibleMrT · Posted: Wed Aug 08, 2012 2:59 pm Post subject:

Hmmm, that didn't make a difference.

It still isn't mapping /dev/mapper/lsi_ehbjieidca4 (lvm partition) when loading the raid - pulls in the other three partitions (boot, swap, root).

This is driving me mad!

I just don't understand why booting the kernel with dmraid (either through liveCD or normal boot) using dmraid would pull in three physical partitions on the disk, but not the fourth.

IncredibleMrT · Posted: Thu Aug 09, 2012 4:03 pm Post subject:

Does anyone have any further ideas? I'm all out.
Thanks.

NeddySeagoon · Posted: Thu Aug 09, 2012 6:14 pm Post subject:

IncredibleMrT,

Tell us how you constructed your raid set, partitioned it and created your LVM phsical volume.

IncredibleMrT · Posted: Fri Aug 10, 2012 8:14 am Post subject:

Hi,
@NeddySeagoon, here is the process - i'm typing from memory, so if things aren't syntactically correct then don't shoot me.

The raid array was set in the BIOS. It was two disks, RAID1. Array created and initialised before liveCD was even put in.

Upon boot, I selected standard gentoo kernel initially - once into the livecd environment I realised that it wasn't showing as raid (nothing in /dev/mapper other than the control file).
Rebooted, this time using 'gentoo dodmraid' as the kernel option.

Now I have /dev/mapper/lsi_ehbjieidca.

This, as far as I was concerned, was all I needed do as far as raid was concerned within the livecd environment. Still saw /dev/sd[ab] but assumed this was normal as long as I used the reference in /dev/mapper

used fdisk /dev/mapper/lsi_ehbjieidca and created boot(83), swap(82), root (83) and lvm(8e)partitions

LVM

pvcreate /dev/mapper/lsi_ehbjieidca
vgcreate vg /dev/mapper/lsi_ehbjieidca
lvcreate -L10G -nusr vg
lvcreate -L2G -nhome vg
lvcreate -L5G -nopt vg
lvcreate -L10G -nvar vg
lvcreate -L2G -ntmp vg

created filesystems on each,
mounted,
etc

Installed as per the handbook, added lvm2, used genkernel (with dmraid & lvm support)to setup kernel (and initramfs) and added statics where needed.

Set grub as follows

NeddySeagoon · Posted: Sat Aug 11, 2012 8:37 pm Post subject:

IncredibleMrT,

I have root on LVM on top of kernel raid5.
I'm fairly sure I hsve not set my partiton type to LVM .. its just linux (83) but I can't check right now,

As you say, seeing /dev/sda, /dev/sdb and /devmapper is quite normal. I think that LVM is looking at /dev/sda4 and /dev/sdb4 separately as you flagged the partition as LVM, then not looking at the raid.

I habe to do all my setup in an initrd, I don't have a lvm.conf at all,

Boot back into the liveCD and set the lvm partition back to 83. lvm will still search for your physical volumes and find them.
From memory vgchange -ay
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

IncredibleMrT · Posted: Mon Aug 13, 2012 9:00 am Post subject:

Tried that @NeddySeagoon - it is no better (or worse)

Still, when it boots under it's own steam (and not a liveCD), it doesn't pick up the fourth partition on that raid array: /dev/mapper/lsi_ehbjieidca4. It finds the other three.
Without this partition, it won't find any lvm detail.

This is the only problem that I can see; if it was able to pick up that partition, then I assume it would find the LVM details and boot up nicely.

:-/

NeddySeagoon · Posted: Mon Aug 13, 2012 6:38 pm Post subject:

IncredibleMrT,

Please dd the first block off of both drives, thats the MBR and email the files to me.

IncredibleMrT · Posted: Tue Aug 14, 2012 8:42 am Post subject:

Morning all,
Here you go Neddy (I did the raided device too just in case):

Right, here they are. I've taken the liberty of doing the same for the raided mapping too. Using dropbox :

/dev/sda -> http://dl.dropbox.com/u/64741464/sdaMBR
/dev/sdb -> http://dl.dropbox.com/u/64741464/sdbMBR
/dev/mapper/lsi_ehbjieidca -> http://dl.dropbox.com/u/64741464/raidedMBR

dmesg -> http://dl.dropbox.com/u/64741464/dmesg.140812

Let me know if you need anything at else

NeddySeagoon · Posted: Tue Aug 14, 2012 7:43 pm Post subject:

IncredibleMrT,

Your files look ok. Rereading ...

IncredibleMrT · Posted: Wed Aug 15, 2012 8:24 am Post subject:

Morning,
The excert from /etc/fstab was incorrect when I first posted it. I noticed and corrected the swap reference to use /dev/mapper/lsi...

the pvcreate & lvcreate commands were both done against /dev/mapper/lsi_ehbjieidca4 - when I added that post I was typing from memory and must have overlooked that detail.

I'm glad you haven't found and glaringly obvious mistakes on my part, so while I feel good about that i'm increasingly frustrated that it doesn't want to play ball.

To that end, having run out of physical machines to use, i'm setting up a virtual gentoo client running off a Win2003 hyper server. Should be adequate for the time being until this problem is put to bed once and for all.

Appreciate all your help

Bring on the genkernel guru!

I'll have a play with a manual kernel too - just started on configuring a fresh seed. I only used genkernel for speed and in the hope it would just work! Oh, and it made setting up and initramfs easier too.
Here's a thought, if I build a manual kernel, can I still use the initramfs that genkernel creates, so:

DaggyStyle · Advocate Joined: 22 Mar 2006 Posts: 4735

small off topic question if I may, assuming that the raid controller is O\B, your rack is quite old and I wonder how does the O\B raid fairs against the kernel's sw raid.
afaik, all O\B raid are sw raid, so you might be better off using the kernel's sw raid which is updated more frequently, probably more stable and will likely to solve your issues.

just a point to take into account if it is even relevant.
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein
ProjectFootball

IncredibleMrT · Posted: Wed Aug 15, 2012 11:01 am Post subject:

I have considered software. The reason I opted for using the hardware's raid because we have a few of these servers and they are reaching the end of their current use - so, if a disk fails, or the machine does, I can just move whatever parts need moving and hopefully it should come up without having to do any tweaks.

That said, I use software raid at home and my business laptop and have found it very stable.

DaggyStyle · Advocate Joined: 22 Mar 2006 Posts: 4735

NeddySeagoon · Posted: Wed Aug 15, 2012 4:41 pm Post subject:

IncredibleMrT,

fakeraid is a version of software raid. Its just that a portion iof the software is provided in your BIOS.
The name fakeraid derives from that fact that it appears as hardware raid to Windows users, who cannot see the underlyiing drives.

For raid1 (and raid0 and raid10) where thare are no parity calculations required, the overhead of software raid (kernel raid or fakeraid) is minimal. Any redundency being provided by writing extra copies of the data. Kernel raid1 scores over fakeraid1 because the kernel positions the heads in different places in different mirror copies in an attempt to reduce latency on reads. i.e. kernel raid is faster in real life as head movements are minimised. fakeraid reads only from a single drive.

Unfortunately, a switch from fakeraid to kernel raid means backing up everthing and reformatting. If you do switch to kernel raid, keep in mind that your cmd64x IDE controller is probably the worst, most buggy design in PC history, so you might want to switch to your serverworks IDE interface too.

I didn't run into any genkernel devs on IRC last nignt, maybe tonight.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

IncredibleMrT · Posted: Fri Aug 17, 2012 8:00 am Post subject:

No rush, i'm running with the gentoo virtual machine on windows hyper-v at the moment, seems quite quick but for the fact I can't seem to get X support - not sure what kernel driver to use for the VGA side - but that's for a separate thread...

Now, onto installing Oracle....

Thanks again for all you help.