| View previous topic :: View next topic |
| Author |
Message |
gr0x0rd
Guru
Joined: 12 Aug 2006
Posts: 311
|
 Posted: Sun Apr 17, 2011 9:24 pm Post subject: Can't install grub on mdadm members (solved) |
 |
|
Hey all,
I'm in the process of rebuilding my desktop/server machine using RAID. I have 4x2TB SATA drives. This is something I have done before, and I've been using my own documentation from my last successful attempt along with various other sources.
After creating my initial partitions and copying the partition table to the other drives, I used the following to set up RAID: | Code: | mdadm --create --verbose /dev/md1 --level=1 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
mdadm --create --verbose /dev/md2 --level=5 --raid-devices=4 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
mdadm --create --verbose /dev/md3 --level=5 --raid-devices=4 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3 |
After mounting the new arrays for installation but before chrooting, I copied over the mdadm.conf as per | Code: | mdadm --detail --scan >> /etc/mdadm.conf
mkdir /mnt/gentoo/etc
cp /etc/mdadm.conf /mnt/gentoo/etc/mdadm.conf |
I can see my arrays are good to go using | Code: | | watch cat /proc/mdstat |
After emerging system utilities, compiling the kernel, and emerging grub, I can't seem to install grub onto any of the member drives. First I tried what was written in my own documentation. But at a grub prompt, | Code: | grub> find /boot/grub/stage1
Error 15: File not found |
. This doesnt seem to make any sense since the file exists in the filesystem: | Code: | livecd ~ # cd /boot/grub
livecd grub # ls -al
total 340
drwxr-xr-x 2 root root 1024 Apr 17 06:37 .
drwxr-xr-x 4 root root 1024 Apr 17 06:37 ..
-rw-r--r-- 1 root root 197 Apr 17 06:37 default
-rw-r--r-- 1 root root 75 Apr 17 06:37 device.map
-rw-r--r-- 1 root root 8772 Apr 17 06:37 e2fs_stage1_5
-rw-r--r-- 1 root root 7932 Apr 17 06:37 fat_stage1_5
-rw-r--r-- 1 root root 7184 Apr 17 06:37 ffs_stage1_5
-rw-r--r-- 1 root root 259 Apr 17 06:38 grub.conf
-rw-r--r-- 1 root root 7192 Apr 17 06:37 iso9660_stage1_5
-rw-r--r-- 1 root root 8644 Apr 17 06:37 jfs_stage1_5
lrwxrwxrwx 1 root root 9 Apr 17 06:37 menu.lst -> grub.conf
-rw-r--r-- 1 root root 7380 Apr 17 06:37 minix_stage1_5
-rw-r--r-- 1 root root 9760 Apr 17 06:37 reiserfs_stage1_5
-rw-r--r-- 1 root root 33856 Apr 17 06:37 splash.xpm.gz
-rw-r--r-- 1 root root 512 Apr 17 06:37 stage1
-rw-r--r-- 1 root root 107736 Apr 17 06:37 stage2
-rw-r--r-- 1 root root 107736 Apr 17 06:37 stage2_eltorito
-rw-r--r-- 1 root root 7452 Apr 17 06:37 ufs2_stage1_5
-rw-r--r-- 1 root root 6804 Apr 17 06:37 vstafs_stage1_5
-rw-r--r-- 1 root root 9576 Apr 17 06:37 xfs_stage1_5 |
When trying the installation as per my documentation, | Code: | grub> device (hd0) /dev/sda
grub> root (hd0,0)
Filesystem type unknown, partition type 0x83
grub> setup (hd0)
Error 17: Cannot mount selected partition |
The method suggested by the gentoo handbook is also unsuccessful: | Code: | livecd grub # grep -v rootfs /proc/mounts > /etc/mtab
livecd grub # grub-install --no-floppy /dev/sda
/dev/md1 does not have any corresponding BIOS drive. |
And yet | Code: | livecd grub # ls -al /dev/sd*
brw-rw---- 1 root disk 8, 0 Apr 17 07:07 /dev/sda
brw-rw---- 1 root disk 8, 1 Apr 17 03:00 /dev/sda1
brw-rw---- 1 root disk 8, 2 Apr 17 03:00 /dev/sda2
brw-rw---- 1 root disk 8, 3 Apr 17 03:00 /dev/sda3
brw-rw---- 1 root disk 8, 16 Apr 17 07:07 /dev/sdb
brw-rw---- 1 root disk 8, 17 Apr 17 03:00 /dev/sdb1
brw-rw---- 1 root disk 8, 18 Apr 17 03:00 /dev/sdb2
brw-rw---- 1 root disk 8, 19 Apr 17 03:00 /dev/sdb3
brw-rw---- 1 root disk 8, 32 Apr 17 07:07 /dev/sdc
brw-rw---- 1 root disk 8, 33 Apr 17 03:00 /dev/sdc1
brw-rw---- 1 root disk 8, 34 Apr 17 03:00 /dev/sdc2
brw-rw---- 1 root disk 8, 35 Apr 17 03:00 /dev/sdc3
brw-rw---- 1 root disk 8, 48 Apr 17 07:07 /dev/sdd
brw-rw---- 1 root disk 8, 49 Apr 17 03:00 /dev/sdd1
brw-rw---- 1 root disk 8, 50 Apr 17 03:00 /dev/sdd2
brw-rw---- 1 root disk 8, 51 Apr 17 03:00 /dev/sdd3 |
Obviously something has changed since last time I set up and documented setting up a RAID1/5 configuration, but my searches aren't turning up anything. I thought maybe this was because grub wasn't able to read an ext2 filesystem, but I just tried again with ext3 and got the same errors.
Any suggestions?
_________________
~g
emerge -FML...
Last edited by gr0x0rd on Mon Apr 18, 2011 7:44 pm; edited 1 time in total |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54214
Location: 56N 3W
|
| Posted: Sun Apr 17, 2011 10:33 pm Post subject: |
|
|
gr0x0rd,
It used to work like a charm but ...
| Code: | | mdadm --create --verbose /dev/md1 --level=1 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 |
now creates version 1.2 raid superblocks by defualt, which are incompatible with grub. You must tell mdadm that you want version 0.9 superblocks.
The bad news gets worse too. The kernel will only auto assemble raid sets with version 0.9 superblocks. You now need an initrd to assemble your root if its on raid with 1.x raid superbocks, since you need to call mdadm before root can be mounted.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
gr0x0rd
Guru
Joined: 12 Aug 2006
Posts: 311
|
| Posted: Mon Apr 18, 2011 5:03 pm Post subject: |
|
|
Thanks for the info sir... too bad it's not good news. At least I know my documentation was correct... 
From what you mentioned, even if I stick with the 1.2 superblocks and set up an initrd for the kernel I'm still screwed because of the grub incompatibility...
Any advice for me? Should I go about rebuilding as before with 0.9 superblocks? I'd imagine compatibility for grub and the kernel regarding the 1.2 superblocks is in development, but I'd probably be waiting a while won't I?
Too funny. Looking at your sig, the EXACT reason I'm putting this array together. Laugh
_________________
~g
emerge -FML... |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54214
Location: 56N 3W
|
| Posted: Mon Apr 18, 2011 6:36 pm Post subject: |
|
|
gr0x0rd,
If you want to use 1.2 superblocks on /boot, you need grub2, which is still hard masked on Gentoo.
You only need revert your /boot partition to raid superblock version 0.9, thats fairly easy. Copy the content somewhere, remake the /boot raid set and copy the content back.
That fixes grub but you still need an initrd to assemble your root.
You can migrate your raid5s to raid superblock is you want. Fail a drive in md2 and md3 (the same drive) and format it without raid, so you can copy everything else over to the single drive. Be sure you remove all traces of the root superblocks. dd /dev/zero to the raid5 partitions is safe.
When you have a single drive copy of your system, destroy the remaining degraded raid5 sets in the same way.
Now you can recreate the raids in degraded mode with version 0.9 superblocks and copy your data back. When you are happy, you can destroy the single drive copy and bring the raid5s up to strength. Its a lot of 'faffing' about.
In this time you could have got an initrd working.
That link may make your eyes glaze over but it contains a good guide to hand rolling a suitable initrd. You can omit the DM-Crypt and LVM2 support if you don't need them.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
gr0x0rd
Guru
Joined: 12 Aug 2006
Posts: 311
|
| Posted: Mon Apr 18, 2011 7:44 pm Post subject: |
|
|
Thanks a lot Neddy.
Since grub2 is still hard masked, going with 0.9 seems like the way to go.
The good news is, I am creating this array with 4 entirely new drives, so I can recreate the RAID5 arrays with 0.9 superblocks with no loss other than my time (I've done it twice already). Something to be said for consistency I guess.
Looks like all I'll need to do is insert when creating the arrays to get things going. Thanks again!
_________________
~g
emerge -FML... |
|
| Back to top |
|
|
Heinz
n00b
Joined: 05 Nov 2009
Posts: 3
Location: Germany,Frankfurt
|
| Posted: Wed Apr 20, 2011 5:44 pm Post subject: |
|
|
| gr0x0rd wrote: | Thanks a lot Neddy.
Since grub2 is still hard masked, going with 0.9 seems like the way to go.
The good news is, I am creating this array with 4 entirely new drives, so I can recreate the RAID5 arrays with 0.9 superblocks with no loss other than my time (I've done it twice already). Something to be said for consistency I guess.
Looks like all I'll need to do is insert when creating the arrays to get things going. Thanks again! |
Töröö,
have the the shit since a week, and "I've been using my own documentation from my last successful attempt along with various other sources" that sound's like me.Anyway, will try it with "--metadata=0.9" too. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Installing Gentoo
