Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Can't install grub on mdadm members (solved)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
gr0x0rd
Guru
Guru


Joined: 12 Aug 2006
Posts: 311

PostPosted: Sun Apr 17, 2011 9:24 pm    Post subject: Can't install grub on mdadm members (solved) Reply with quote

Hey all,

I'm in the process of rebuilding my desktop/server machine using RAID. I have 4x2TB SATA drives. This is something I have done before, and I've been using my own documentation from my last successful attempt along with various other sources.

After creating my initial partitions and copying the partition table to the other drives, I used the following to set up RAID:
Code:
mdadm --create --verbose /dev/md1 --level=1 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
mdadm --create --verbose /dev/md2 --level=5 --raid-devices=4 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
mdadm --create --verbose /dev/md3 --level=5 --raid-devices=4 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3
After mounting the new arrays for installation but before chrooting, I copied over the mdadm.conf as per
Code:
mdadm --detail --scan >> /etc/mdadm.conf
mkdir /mnt/gentoo/etc
cp /etc/mdadm.conf /mnt/gentoo/etc/mdadm.conf
I can see my arrays are good to go using
Code:
watch cat /proc/mdstat
After emerging system utilities, compiling the kernel, and emerging grub, I can't seem to install grub onto any of the member drives. First I tried what was written in my own documentation. But at a grub prompt,
Code:
grub> find /boot/grub/stage1

Error 15: File not found
. This doesnt seem to make any sense since the file exists in the filesystem:
Code:
livecd ~ # cd /boot/grub
livecd grub # ls -al
total 340
drwxr-xr-x 2 root root   1024 Apr 17 06:37 .
drwxr-xr-x 4 root root   1024 Apr 17 06:37 ..
-rw-r--r-- 1 root root    197 Apr 17 06:37 default
-rw-r--r-- 1 root root     75 Apr 17 06:37 device.map
-rw-r--r-- 1 root root   8772 Apr 17 06:37 e2fs_stage1_5
-rw-r--r-- 1 root root   7932 Apr 17 06:37 fat_stage1_5
-rw-r--r-- 1 root root   7184 Apr 17 06:37 ffs_stage1_5
-rw-r--r-- 1 root root    259 Apr 17 06:38 grub.conf
-rw-r--r-- 1 root root   7192 Apr 17 06:37 iso9660_stage1_5
-rw-r--r-- 1 root root   8644 Apr 17 06:37 jfs_stage1_5
lrwxrwxrwx 1 root root      9 Apr 17 06:37 menu.lst -> grub.conf
-rw-r--r-- 1 root root   7380 Apr 17 06:37 minix_stage1_5
-rw-r--r-- 1 root root   9760 Apr 17 06:37 reiserfs_stage1_5
-rw-r--r-- 1 root root  33856 Apr 17 06:37 splash.xpm.gz
-rw-r--r-- 1 root root    512 Apr 17 06:37 stage1
-rw-r--r-- 1 root root 107736 Apr 17 06:37 stage2
-rw-r--r-- 1 root root 107736 Apr 17 06:37 stage2_eltorito
-rw-r--r-- 1 root root   7452 Apr 17 06:37 ufs2_stage1_5
-rw-r--r-- 1 root root   6804 Apr 17 06:37 vstafs_stage1_5
-rw-r--r-- 1 root root   9576 Apr 17 06:37 xfs_stage1_5
When trying the installation as per my documentation,
Code:
grub> device (hd0) /dev/sda

grub> root (hd0,0)
 Filesystem type unknown, partition type 0x83

grub> setup (hd0)

Error 17: Cannot mount selected partition
The method suggested by the gentoo handbook is also unsuccessful:
Code:
livecd grub # grep -v rootfs /proc/mounts > /etc/mtab
livecd grub # grub-install --no-floppy /dev/sda
/dev/md1 does not have any corresponding BIOS drive.
And yet
Code:
livecd grub # ls -al /dev/sd*
brw-rw---- 1 root disk 8,  0 Apr 17 07:07 /dev/sda
brw-rw---- 1 root disk 8,  1 Apr 17 03:00 /dev/sda1
brw-rw---- 1 root disk 8,  2 Apr 17 03:00 /dev/sda2
brw-rw---- 1 root disk 8,  3 Apr 17 03:00 /dev/sda3
brw-rw---- 1 root disk 8, 16 Apr 17 07:07 /dev/sdb
brw-rw---- 1 root disk 8, 17 Apr 17 03:00 /dev/sdb1
brw-rw---- 1 root disk 8, 18 Apr 17 03:00 /dev/sdb2
brw-rw---- 1 root disk 8, 19 Apr 17 03:00 /dev/sdb3
brw-rw---- 1 root disk 8, 32 Apr 17 07:07 /dev/sdc
brw-rw---- 1 root disk 8, 33 Apr 17 03:00 /dev/sdc1
brw-rw---- 1 root disk 8, 34 Apr 17 03:00 /dev/sdc2
brw-rw---- 1 root disk 8, 35 Apr 17 03:00 /dev/sdc3
brw-rw---- 1 root disk 8, 48 Apr 17 07:07 /dev/sdd
brw-rw---- 1 root disk 8, 49 Apr 17 03:00 /dev/sdd1
brw-rw---- 1 root disk 8, 50 Apr 17 03:00 /dev/sdd2
brw-rw---- 1 root disk 8, 51 Apr 17 03:00 /dev/sdd3
Obviously something has changed since last time I set up and documented setting up a RAID1/5 configuration, but my searches aren't turning up anything. I thought maybe this was because grub wasn't able to read an ext2 filesystem, but I just tried again with ext3 and got the same errors.

Any suggestions?
_________________
~g
emerge -FML...


Last edited by gr0x0rd on Mon Apr 18, 2011 7:44 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 39636
Location: 56N 3W

PostPosted: Sun Apr 17, 2011 10:33 pm    Post subject: Reply with quote

gr0x0rd,

It used to work like a charm but ...
Code:
mdadm --create --verbose /dev/md1 --level=1 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
now creates version 1.2 raid superblocks by defualt, which are incompatible with grub. You must tell mdadm that you want version 0.9 superblocks.

The bad news gets worse too. The kernel will only auto assemble raid sets with version 0.9 superblocks. You now need an initrd to assemble your root if its on raid with 1.x raid superbocks, since you need to call mdadm before root can be mounted.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
gr0x0rd
Guru
Guru


Joined: 12 Aug 2006
Posts: 311

PostPosted: Mon Apr 18, 2011 5:03 pm    Post subject: Reply with quote

Thanks for the info sir... too bad it's not good news. At least I know my documentation was correct... :)

From what you mentioned, even if I stick with the 1.2 superblocks and set up an initrd for the kernel I'm still screwed because of the grub incompatibility...

Any advice for me? Should I go about rebuilding as before with 0.9 superblocks? I'd imagine compatibility for grub and the kernel regarding the 1.2 superblocks is in development, but I'd probably be waiting a while won't I?

Too funny. Looking at your sig, the EXACT reason I'm putting this array together. Laugh
_________________
~g
emerge -FML...
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 39636
Location: 56N 3W

PostPosted: Mon Apr 18, 2011 6:36 pm    Post subject: Reply with quote

gr0x0rd,

If you want to use 1.2 superblocks on /boot, you need grub2, which is still hard masked on Gentoo.
You only need revert your /boot partition to raid superblock version 0.9, thats fairly easy. Copy the content somewhere, remake the /boot raid set and copy the content back.

That fixes grub but you still need an initrd to assemble your root.

You can migrate your raid5s to raid superblock is you want. Fail a drive in md2 and md3 (the same drive) and format it without raid, so you can copy everything else over to the single drive. Be sure you remove all traces of the root superblocks. dd /dev/zero to the raid5 partitions is safe.
When you have a single drive copy of your system, destroy the remaining degraded raid5 sets in the same way.

Now you can recreate the raids in degraded mode with version 0.9 superblocks and copy your data back. When you are happy, you can destroy the single drive copy and bring the raid5s up to strength. Its a lot of 'faffing' about.

In this time you could have got an initrd working.
That link may make your eyes glaze over but it contains a good guide to hand rolling a suitable initrd. You can omit the DM-Crypt and LVM2 support if you don't need them.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
gr0x0rd
Guru
Guru


Joined: 12 Aug 2006
Posts: 311

PostPosted: Mon Apr 18, 2011 7:44 pm    Post subject: Reply with quote

Thanks a lot Neddy.

Since grub2 is still hard masked, going with 0.9 seems like the way to go.

The good news is, I am creating this array with 4 entirely new drives, so I can recreate the RAID5 arrays with 0.9 superblocks with no loss other than my time (I've done it twice already). Something to be said for consistency I guess.

Looks like all I'll need to do is insert
Code:
--metadata=0.9
when creating the arrays to get things going. Thanks again!
_________________
~g
emerge -FML...
Back to top
View user's profile Send private message
Heinz
n00b
n00b


Joined: 05 Nov 2009
Posts: 3
Location: Germany,Frankfurt

PostPosted: Wed Apr 20, 2011 5:44 pm    Post subject: Reply with quote

gr0x0rd wrote:
Thanks a lot Neddy.

Since grub2 is still hard masked, going with 0.9 seems like the way to go.

The good news is, I am creating this array with 4 entirely new drives, so I can recreate the RAID5 arrays with 0.9 superblocks with no loss other than my time (I've done it twice already). Something to be said for consistency I guess.

Looks like all I'll need to do is insert
Code:
--metadata=0.9
when creating the arrays to get things going. Thanks again!



Töröö,
have the the shit since a week, and "I've been using my own documentation from my last successful attempt along with various other sources" that sound's like me.Anyway, will try it with "--metadata=0.9" too.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum