raid0, raid1 and LVM2 install issues ... PLEASE HELP

[jeastman]

I hope this finds you doing well. I am trying to install 2007.0 with software raid0 and raid1 using LVM2 for logical volume management and I'm having quite a rough time. I've followed the instructions from the Gentoo documentation for doing software raid/LVM installs found at http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml. Everything about the install starts just fine. I go through the fdisk on both my 80GB WesternDigital Caviar drives and end up with disk partitions that are exactly the same. Here they are ->

livecd ~ # fdisk -l /dev/sda

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 12 96358+ fd Linux raid autodetect
/dev/sda2 13 262 2008125 82 Linux swap / Solaris
/dev/sda3 263 387 1004062+ fd Linux raid autodetect
/dev/sda4 388 9729 75039615 fd Linux raid autodetect
livecd ~ # fdisk -l /dev/sdb

Disk /dev/sdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 * 1 12 96358+ fd Linux raid autodetect
/dev/sdb2 13 262 2008125 82 Linux swap / Solaris
/dev/sdb3 263 387 1004062+ fd Linux raid autodetect
/dev/sdb4 388 9729 75039615 fd Linux raid autodetect
livecd ~ #

Next I spin up the swap partitions with the command swapon -p 1 /dev/sda2 && swapon -p 1 /dev/sdb2. To make sure they are both up I run the following ->

livecd ~ # swapon -v -s
Filename Type Size Used Priority
/dev/sda2 partition 2008116 0 1
/dev/sdb2 partition 2008116 0 1

I then create my raid nodes in /dev and create my raid devices with ->

livecd ~ # mknod /dev/md1 b 9 1
livecd ~ # mknod /dev/md3 b 9 3
livecd ~ # mknod /dev/md4 b 9 4

livecd ~ # mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm: array /dev/md1 started.
livecd ~ # mdadm --create /dev/md3 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3
mdadm: array /dev/md3 started.
livecd ~ # mdadm --create /dev/md4 --level=0 --raid-devices=2 /dev/sda4 /dev/sdb4
mdadm: array /dev/md4 started.

When I run -> livecd ~ # cat /proc/mdstat I see ->

livecd ~ # cat /proc/mdstat
Personalities : [raid1] [raid0]
md4 : active raid0 sdb4[1] sda4[0]
150078976 blocks 64k chunks

md3 : active raid1 sdb3[1] sda3[0]
1003968 blocks [2/2] [UU]

md1 : active raid1 sdb1[1] sda1[0]
96256 blocks [2/2] [UU]

unused devices: <none>

So, right now I feel pretty good. Now I bring my logical volumes up and see that they are available. I do this with ->

livecd ~ # vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg" using metadata type lvm2
livecd ~ # vgchange -a y
8 logical volume(s) in volume group "vg" now active
livecd ~ # vgs
VG #PV #LV #SN Attr VSize VFree
vg 1 8 0 wz--n- 143.12G 103.12G
livecd ~ # lvs
LV VG Attr LSize Origin Snap% Move Log Copy%
distfiles vg -wi-a- 4.00G
home vg -wi-a- 10.00G
opt vg -wi-a- 4.00G
portage vg -wi-a- 2.00G
tmp vg -wi-a- 2.00G
usr vg -wi-a- 8.00G
var vg -wi-a- 4.00G
vartmp vg -wi-a- 6.00G

I now mount my file systems, including proc and dev and am ready to chroot. Here's my df -h ->

livecd ~ # mount -t proc none /mnt/gentoo/proc
livecd ~ # mount -o bind /dev /mnt/gentoo/dev
livecd ~ # df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 506M 4.4M 501M 1% /
/dev/hda 58M 58M 0 100% /mnt/cdrom
/dev/loop0 44M 44M 0 100% /mnt/livecd
udev 506M 652K 505M 1% /dev
tmpfs 506M 2.1M 504M 1% /mnt/livecd/lib/firmware
tmpfs 506M 0 506M 0% /mnt/livecd/usr/portage
/dev/md3 965M 185M 732M 21% /mnt/gentoo
/dev/md1 92M 7.1M 80M 9% /mnt/gentoo/boot
/dev/mapper/vg-usr 7.9G 931M 6.6G 13% /mnt/gentoo/usr
/dev/mapper/vg-home 9.9G 151M 9.2G 2% /mnt/gentoo/home
/dev/mapper/vg-opt 4.0G 137M 3.7G 4% /mnt/gentoo/opt
/dev/mapper/vg-tmp 2.0G 3.0M 1.9G 1% /mnt/gentoo/tmp
/dev/mapper/vg-var 4.0G 252M 3.5G 7% /mnt/gentoo/var
/dev/mapper/vg-vartmp
6.0G 15M 5.6G 1% /mnt/gentoo/var/tmp
/dev/mapper/vg-portage
2.0G 237M 1.7G 13% /mnt/gentoo/usr/portage
/dev/mapper/vg-distfiles
4.0G 79M 3.8G 3% /mnt/gentoo/usr/portage/distfiles

I've downloaded the stage and portage tars and now I chroot ->

livecd ~ # chroot /mnt/gentoo /bin/bash
livecd / # env-update && source /etc/profile
>>> Regenerating /etc/ld.so.cache...
livecd / # df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 965M 185M 732M 21% /
/dev/hda 965M 185M 732M 21% /mnt/cdrom
df: `/mnt/livecd': No such file or directory
sysfs 965M 185M 732M 21% /sys
udev 506M 652K 505M 1% /dev
devpts 506M 652K 505M 1% /dev/pts
df: `/mnt/livecd/lib/firmware': No such file or directory
df: `/mnt/livecd/usr/portage': No such file or directory
/dev/md3 965M 185M 732M 21% /
/dev/md1 92M 7.1M 80M 9% /boot
/dev/vg/usr 7.9G 931M 6.6G 13% /usr
/dev/vg/home 9.9G 151M 9.2G 2% /home
/dev/vg/opt 4.0G 137M 3.7G 4% /opt
/dev/vg/tmp 2.0G 3.0M 1.9G 1% /tmp
/dev/vg/var 4.0G 252M 3.5G 7% /var
/dev/vg/vartmp 6.0G 15M 5.6G 1% /var/tmp
/dev/vg/portage 2.0G 237M 1.7G 13% /usr/portage
/dev/vg/distfiles 4.0G 79M 3.8G 3% /usr/portage/distfiles
udev 506M 652K 505M 1% /dev

From all of this I notice that my file systems survived the chroot and my /dev/md1, /dev/md3 and /dev/md4 is available. Once in the chroot I see that my /etc/lvm/lvm.conf is there and has the all important line saying ->

filter = [ "a|/dev/sd[ab]|", "r/.*/" ]

I also take a look at my /etc/mdadm.conf and I see ->

DEVICE /dev/sda*
DEVICE /dev/sdb*
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=67b85a0e:c4866847:6848d0a3:c42a1837
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=5b6bda4b:67d9f255:63b63c35:99d3e1e6
ARRAY /dev/md4 level=raid0 num-devices=2 UUID=962610df:8cfe6091:edbc452e:733b1715
MAILADDR root@localhost

I now do various emerges and I build my kernel using genkernel with the DM and LVM stuff built in to the kernel along with all the file systems I need/want. In other words, my kernel is monolithic and no modules. So ... now I do the grub part. I start with ->

grub> find /boot/grub/stage1
(hd0,0)
(hd1,0)

grub> device (hd0) /dev/sda

grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded.
succeeded
Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
Done.

grub> device (hd0) /dev/sdb

grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded.
succeeded
Running "install /boot/grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"... succeeded
Done.

grub>

Here's what my grub.conf looks like ->

default 0
timeout 10
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title=Gentoo Linux 2.6.20-r8
root (hd0,0)
kernel /boot/kernel-genkernel-x86-2.6.20-gentoo-r8 udev dodmraid dolvm2 lvmraid=/dev/md3 root=/dev/md3 md=3,/dev/sda3,/dev/sdb3 init=/linuxrc ramdisk=8192
initrd /boot/initramfs-genkernel-x86-2.6.20-gentoo-r8

At this point I believe all is well. But, this is not so. Here's what I see on reboot.....

After I reboot my newly installed machine I see it go to the Gentoo splash screen and I choose the kernel I installed. It starts to boot and gets to the part where it loads the device mapper for the raid volumes and I see ->

>> Loading Modules
:: Scanniong for dm-mirror ... dm-mirror loaded.
:: Scanning for nfs ... nfs loaded.
:: Scanning for dm-snapshot .. dm-snapshot loaded.
>> Activating mdev
>> Detected realP_root as a md device. Setting up the device node...
>> Detected realP_root as a md device. Setting up the device node...
>> Scanning for Volume Groups
Reading all physical volumes. This may take a while...
>> Activating Volume Groups
>> Determining root device...
>> Mounting root
mount: Mounting /dev/md3 on /newroot failed: No such file or directory
!! Could not moutn specified ROOT, try again
!! Teh root block device is unspecified or not detected.
Please specify a device to boot, or "shell" fora shell...
boot()::

So, I type shell and I get a shell .... as you might expect. When I do ls -la /dev I see that there is a /dev/md3. However, if I do more/dev/md3 I see nothing. I find this odd as there was something in that file after I did the mknod from the installation instructions. What am I doing wrong? Any help you can provide is much appreciated.

jeastman

[HeissFuss]

You don't need to specify md=3,/dev/sda3,/dev/sdb3, nor put it in your /etc/mdadm.conf since it will be autodetected.

I've never had much luck with genkernel, but make sure that you build it with mdraid support.

[dalu]

I've got a similar, if not the same problem. not sure if i should create a new topic since people tend to answer new topics more quickly that posting in existing ones.

I've more/less done what the OP described at the beginning of this year on a hetzner ds3000 server and it went well, worked on 1st try (i didn't append the dev/sda3 stuff to grub.conf/menu.lst)

[dalu]

ok sorry, since i've now included all the s-ata drivers into the kernel the error message changed

posting what i can see through remote adapter

[dalu]

solution was