Grub and RAID install, HELP

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

i have a gentoo install setup like this.

i have 4 1TB disks, i used the fdisk utility to partition each identically, no special configuration.

so now have
/dev/sda1 /dev/sda2 /dev/sda3 /dev/sda5 /dev/sdd6
/dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdb5 /dev/sdd6
/dev/sdc1 /dev/sdc2 /dev/sdc3 /dev/sdc5 /dev/sdd6
/dev/sdd1 /dev/sdd2 /dev/sdd3 /dev/sdd5 /dev/sdd6

and used mdadm like this

mdadm --create --verbose /dev/md1 --level=1 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
mdadm --create --verbose /dev/md2 --level=5 --raid-devices=4 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
mdadm --create --verbose /dev/md3 --level=5 --raid-devices=4 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3
mdadm --create --verbose /dev/md4 --level=5 --raid-devices=4 /dev/sda5 /dev/sdb5 /dev/sdc5 /dev/sdd5
mdadm --create --verbose /dev/md5 --level=5 --raid-devices=4 /dev/sda6 /dev/sdb6 /dev/sdc6 /dev/sdd6

works fine.

i mounted, installed like normal, and even installed grub on each of the 4 disks

my grub.conf

Mike Hunt · Watchman Joined: 19 Jul 2009 Posts: 5287

I think these are relevant:

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

ok that did not help me at all, since those are the directions that i followed in the first place.

i did get gentoo to boot into the kernel, my problem was grub wasn't able to find /boot/kernel because my /boot partition was missing the "boot" symlink, so i just added that symlink in again and it loaded the kernel.

now my new problem is a kernel panic, it said md3 was not a valid root partition, enter a real partition to root=
available partitions are:

what is the correct way to configure my kernel in a way that would recognize and assemble the /dev/md3 partition ?
_________________
RemixTechnology.com
help the needy

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

In my kernel config, i enabled "autodetect RAID at boot"
the kernel panic error says:

cyrillic · Posted: Fri Feb 05, 2010 12:39 am Post subject:

Are your partitions marked "type fd" Linux raid autodetect, instead of "type 83" Linux ?

Mike Hunt · Watchman Joined: 19 Jul 2009 Posts: 5287

Ah yes, a raid51 - never did one.

Generally, unknown-block(0,0) means that the hardware isn't detectected.

Do you have DM_MIRROR=y ?

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

before

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

failed.
i got the exact same kernel panic message

after i changed the partition types, was i supposed to rebuild the raid array? or assemble then reformat the filesystem?

all i did was change the partition types, assembled the array, mounted and chrooted so that i can update the kernel to add DM_MIRROR=y
recompiled, copied to /boot

even did grub setup again on each disk.

and then rebooted

any other ideas, or, how can i find out what i'm doing wrong?
_________________
RemixTechnology.com
help the needy

Mike Hunt · Watchman Joined: 19 Jul 2009 Posts: 5287

Nevermind about DM_MIRROR=y, it's for LVM.

But, don't you really need 2 raid1 drives - a mirrored pair? Pretty sure you do.
In fact I think you need a minimum of 6 drives to raid1 + raid5.
Interesting article.

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

actually i don't have a 5+1 or 1+5

i just have 4 1TB disks
partition 1 = RAID 1 (boot partition)
partition 2 = RAID 5
partition 3 = RAID 5
partition 4 = extended
partition 5 = RAID 5
partition 6 = RAID 5

so it seems the kernel isn't autodetecting any of the RAIDs and it errors out because the root device (/dev/md3) doesn't exist.

i have NVRaid, and have that turned off. is there any way i can get RAID5 to work on linux?
_________________
RemixTechnology.com
help the needy

Mike Hunt · Watchman Joined: 19 Jul 2009 Posts: 5287

I might consider something like this, just as an example:

NeddySeagoon · Posted: Fri Feb 05, 2010 8:19 pm Post subject:

remix,

This works for me :-

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

hi NeddySeagoon,

that's what i have, except

root (hd0,0)
kernel /boot/2.6.31-gentoo-r5 root=/dev/md3

works for me, as in, in loads the kernel just fine, my problem is when the kernel loads, it doesn't autodetect my raid5.

could you please share your kernel config? is there a special kernel version that i must use?

thanks for your help
_________________
RemixTechnology.com
help the needy

cyrillic · Posted: Sat Feb 06, 2010 12:54 am Post subject:

Do your arrays have version 0.9 superblocks ?

I don't think the kernel's autodetection works with version 1.x superblocks.

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

yes 0.9

NeddySeagoon · Posted: Sat Feb 06, 2010 11:07 am Post subject:

remix,

Here are some of my kernel configs They are all gentoo-sources, even though the names differ.

As your other thread talks about dmraid and turning on raid in the BIOS, which raid system are you using. Pick just one.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

cyrillic · Posted: Sun Feb 07, 2010 2:44 pm Post subject:

I guess the one thing that remains is to make sure your kernel has the correct driver(s) needed to access the controller(s) that your individual harddrives are plugged into.

If the kernel doesn't see the drives, then the kernel will not be able to assemble the array(s).

remix · l33t Joined: 28 Apr 2004 Posts: 768 Location: hawaii

Thanks Neddy. i wanted to use your kernel configs, and before i could complete emerging your kernel version(s) i get kernel panics from the raid Pid:7102, comm: md3_raid5 Not tainted 2.6.30-gentoo-r5 #1 System Product Name

so i rebuilt the raid using mdraid --create and reformatted the raid device in ext3
and when i extrated the stage tarball into it, i got another kernel panic.

so i rebuilt it again, and i was able to go all the way into chrooting, and i wasn't able to even emerge or build a kernel before getting a kernel panic.

is this a serious problem ?

is there a way to fix this or did i just waste hundreds of bucks on brand new 1TB drives

_________________
RemixTechnology.com
help the needy

NeddySeagoon · Posted: Wed Feb 10, 2010 8:56 pm Post subject:

remix,

I don't see it being a CD or a hard drive issue - I put some advice in your other thread.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

eccerr0r · Advocate Joined: 01 Jul 2004 Posts: 2354 Location: USA

I use a 4-disk RAID5, boot-off-same disks (4-way RAID1).

I'm currently using the old ATA drivers. This way works well, my software md RAID5 root is autodetected.

What I'm currently fearing when I do migrate to SATA is similar to the issue with root USB disks - whether the SATA disks will enumerate in time before it checks for LVM/md partitions -- in this case, the only way to ensure that it detects in the right order is through initrd... Looking at your dmesg, is it autosearching for raid disks before all the SATA disks are detected?
_________________
Core2Quad 9550S/4GB/4x500G RAID5/RadeonHD 5770
What the heck am I advocating?

cyrillic · Posted: Sat Feb 13, 2010 12:46 am Post subject:

tp11235 · n00b Joined: 22 Aug 2006 Posts: 35 Location: England

Hope you don't mind me joining in, but this thread seems very relevant to my problem.

I have run Gentoo for a few years now, but this is my first venture into software RAID. I get as far as the reboot but GRUB hangs with "GRUB Stage 1_5" on the screen.

My setup is as follows:

Disk /dev/sda: 164.7 GB, 164696555520 bytes
255 heads, 63 sectors/track, 20023 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x552b1665

Device Boot Start End Blocks Id System
/dev/sda1 * 1 9 72261 83 Linux
/dev/sda2 10 141 1060290 82 Linux swap / Solaris
/dev/sda3 142 20023 159702165 fd Linux raid autodetect

Disk /dev/sdb: 164.7 GB, 164696555520 bytes
255 heads, 63 sectors/track, 20023 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xa78dab71

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 9 72261 83 Linux
/dev/sdb2 10 141 1060290 82 Linux swap / Solaris
/dev/sdb3 142 20023 159702165 fd Linux raid autodetect

I know that having root on a RAID0 is "not recommended". My first question is whether this is because it is dangerous (not an issue for me on this machine) or because it does not work at all.

The following shows me bringing my array back to life after a reboot during installation:

livecd ~ # mknod /dev/md1 b 9 1
livecd ~ # mknod /dev/md3 b 9 3
livecd ~ # mdadm -A /dev/md1 /dev/sda1 /dev/sdb1
mdadm: /dev/md1 has been started with 2 drives.
livecd ~ # mdadm -A /dev/md3 /dev/sda3 /dev/sdb3
mdadm: /dev/md3 has been started with 2 drives.
livecd ~ # vgscan --mknodes
Reading all physical volumes. This may take a while...
Found volume group "vg" using metadata type lvm2
livecd ~ # vgchange -a y
1 logical volume(s) in volume group "vg" now active
livecd ~ # mount /dev/vg/topsy /mnt/gentoo
livecd ~ # mount /dev/md1 /mnt/gentoo/boot
livecd ~ # lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
topsy vg -wi-ao 304.00g
livecd ~ # vgs
VG #PV #LV #SN Attr VSize VFree
vg 1 1 0 wz--n- 304.61g 620.00m

/dev/md1 is RAID1 and /dev/md3 is RAID0

This is my FSTAB

/dev/md1 /boot ext2 noauto,noatime 1 2
/dev/vg/topsy / ext3 noatime 0 1
/dev/sda2 none swap sw,pri=1 0 0
/dev/sdb2 none swap sw,pri=1 0 0

/dev/cdrom /mnt/cdrom auto noauto,ro 0 0

This is my grub.conf

default 0
timeout 30
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title Gentoo Linux 2.6.24-r6 (hd0)
root (hd0,0)
kernel /boot/kernel-2.6.31-gentoo-r6 root=/dev/vg/topsy

title Gentoo Linux 2.6.24-r6 (hd1)
root (hd1,0)
kernel /boot/kernel-2.6.31-gentoo-r6 root=/dev/vg/topsy

I have been following the Gentoo LVM2 Installation Guide at http://www.gentoo.org/doc/en/lvm2.xml as well as the quickstart guide http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml.

I made up the bit in grub.conf with the root=/dev/vg/topsy because it seemed likely that it needed to refer to the logical volume rather than the /dev/md3 device node. I tried both.

I saw some stuff about deleting the 1_5 bits from grub - but I really need to understand more about how grub works with a raid. I get that /boot and so GRUB has to be in RAID1 because the BIOS can only read unstriped. Then GRUB needs the kernel in the same unstriped partition. However, can the kernel then read config files in root on a striped partition from the outset so that it can start the system?

I emerged LVM2, mdadm and set the appropriate options when I compiled the kernel.

Any help and advice gratefully received.

Regards

Tim.

cyrillic