[SOLVED]-Cant install grub to raid10 /boot

Message

FizzyWidget · Post by **FizzyWidget** » Wed Nov 28, 2012 10:28 am

okay so this is where i feel down before with grub, according to the wiki

http://en.gentoo-wiki.com/wiki/RAID/Software

Since the /boot partition is a RAID, grub cannot read it to get the bootloader. It can only access physical drives. Thus, you still use (hd0,0) in this step.
Run grub:
grub --no-floppy
You must see GRUB prompt:
grub>

If you are using a RAID 1 mirror disk system, you will want to install grub on all the disks in the system, so that when one disk fails, you are still able to boot. The find command above will list the disks, e.g.
grub> find /boot/grub/stage1
 (hd0,0)
 (hd1,0)
grub>

Code: Select all

and this is what i get

grub> find /boot/grub/stage1

Error 15: File not found


also tried


grub> find /boot/grub/stage1
Error 12: Invalid device requested

grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... no

Error 15: File not found

Code: Select all

ls /boot/grub
default     e2fs_stage1_5  ffs_stage1_5  iso9660_stage1_5  menu.lst        reiserfs_stage1_5  stage1  stage2_eltorito  vstafs_stage1_5
device.map  fat_stage1_5   grub.conf     jfs_stage1_5      minix_stage1_5  splash.xpm.gz      stage2  ufs2_stage1_5    xfs_stage1_5

should i just be pointing it to /dev/grub ?

edit: seems grub (legacy) has issues with raid10, should I try grub2 or break md0 and make it raid1?

gabrielg · Post by **gabrielg** » Wed Nov 28, 2012 4:47 pm

What superblock version are you using? I remember reading that only with 0.9x it works [with mdadm]. Not sure which version of grub you should use for the job, maybe you're right and you need 2 (which is 1.99 really).

I didn't even bother personally, I just stuck a USB drive where I've got my boot partition and live happily with a full disk RAID 10.

FizzyWidget · Post by **FizzyWidget** » Wed Nov 28, 2012 5:29 pm

I am thinking of doing that as no matter if i try grub or grub2 it doesnt want to work, I have trid md0 /dev/sda and all gives me errors, from what i have been able to google my only other option is to go raid1 it might be that it doesnt like raid10

Post by **NeddySeagoon** » Wed Nov 28, 2012 8:37 pm

Dark Foo,

Grub cannot read raid at all - its just ignored but you can use raid1 for boot if you wish as long as you also use raid superblock version 0.90.
You might get grub to play on raid10 too as long as it did not need to cross stripes, which is I suppose unlikely.

FizzyWidget · Post by **FizzyWidget** » Wed Nov 28, 2012 8:40 pm

yes I think raid10 is the issue, I am using -e0.90 when i make the raid, I will redo the whole system tomorrow, and make /boot raid1 - can you use 4 drives in a raid1 or is it just 2?

Post by **NeddySeagoon** » Wed Nov 28, 2012 8:43 pm

Dark Foo,

With version 0.90 superblocks, the limit is 28 elements to a raid set.
I use a maximum of 5 so far.

FizzyWidget · Post by **FizzyWidget** » Wed Nov 28, 2012 8:52 pm

Just so that I am sure, i should install grub to all 4 drives and not only the first two as it says in the wiki?

Post by **NeddySeagoon** » Wed Nov 28, 2012 9:20 pm

Dark Foo,

Correct. Raid10 is proof against all single drive failures and some combinations of two failed drives. To be proof against all combinations of two filed drives, use raid6. Raid6 will not give you the speed improvements you hope to get from raid10.
To be sure you can boot in the face of a usable system with two failed drives, you need grub on 3 or more drives in your 4 drive raid10.

Use setup (hdx,y) and point grub to its own copy of /boot. When grub runs at boot time, its just 4 drives with identical content. The raid set is not formed until after the kernel is loaded, by which time grub is no longer running.
In any case, grub stage1 and grub stage1.5 are installed outside of the filesystem space and are therefore outside of the raided area

FizzyWidget · Post by **FizzyWidget** » Wed Nov 28, 2012 9:22 pm

my board cant do raid6 so I will go with raid1 for /boot and raid10 for the rest

Post by **NeddySeagoon** » Wed Nov 28, 2012 9:27 pm

Dark Foo,

Are you not using kernel raid and mdadm?

FizzyWidget · Post by **FizzyWidget** » Wed Nov 28, 2012 9:28 pm

yes raid is set in the kernel and I am using mdadm, oh brain fart, i can use raid6 as i am not using the on-board raid - d'oh

So raid1 for /boot raid6 for the others, I will have a play about tomorrow and see what happens

Post by **NeddySeagoon** » Wed Nov 28, 2012 10:06 pm

Dark Foo,

It depends what you want from your raid, speed, reliability or some of both.
Raid6 spreads redundant information across N drives so the data can be recovered from any N-2. There is a CPU overhead in calculating this data on write and decoding it on read.
Raid10, is raid0 on top of a pair of raid1 sets, (or the other way up, I forget which). There is no redundant data to calculate or decode, raid1 and raid10 write multiple copies of the same data that would have been written to single drive, on read, only one drive needs to be read at a time but you get a speedup as all four can be read in parallel. I'm not sure if the kernel actually does this or not. Compared to a single drive, writes are slower on raid, except raid0, and reads may be faster.

FizzyWidget · Post by **FizzyWidget** » Wed Nov 28, 2012 10:07 pm

basically its for storage and streaming to my ps3, might using it as a render node for a few windows 3d apps - so speed is important, but so is making sure my data is safe

Post by **NeddySeagoon** » Wed Nov 28, 2012 10:20 pm

Dark Foo,

Raid is not a substitute for backups. Its a reliability improvement when you have a lot of drives in use.

If an individual drive has a failure rate of F, then failure rate for a drive in a raid set is N*F. You have more drives so more opportunities for a failure.
The Redundant aspect (the r in raid) combats the increased failure rate.

FizzyWidget · Post by **FizzyWidget** » Thu Nov 29, 2012 11:28 pm

well i got further on raid1 than i did with raid10, but now i am hitting an error of

switch_root chdir/newroot no such directory

here is my init

Code: Select all

#!/bin/busybox sh 
mount -t proc proc /proc 
mount -t sysfs sysfs /sys 
mount -t devtmpfs devtmpfs /dev 
#for a french azerty keyboard 
loadkmap < /etc/kmap-us 
rescue() { 
   echo "Dropping to rescue shell" >&2 
   /bin/sh </dev/tty1 >/dev/tty1 2>&1 
} 

# assemble root 
/bin/mdadm --assemble /dev/md1 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2 
# unlock root 
/bin/cryptsetup luksOpen /dev/md1 gentoo-root || rescue 
# mount root ro 
mount -o ro /dev/mapper/gentoo-root / || rescue 

CMDLINE=`cat /proc/cmdline` 
umount /dev 
umount /sys 
umount /proc 
exec /bin/busybox switch_root /newroot /sbin/init ${CMDLINE}

can anyone see anything out of the ordinary?

It asks for pass, which i give it then is throws up a mount error and then the no such dir issue

cach0rr0 · Post by **cach0rr0** » Fri Nov 30, 2012 7:12 am

this

Code: Select all

mount -o ro /dev/mapper/gentoo-root / || rescue

is not correct. remember / is in ram until you switch_root, so in the above you are telling the initramfs to mount your rootfs over the top of it, instead of at a different mount point where it wont clobber things

steps:
-go back to the root of your initramfs structure, and do a 'mkdir -p mnt/root'. This directory will be empty, but this will be where you mount gentoo-root
-edit your 'init' to be

Code: Select all

mount -o ro /dev/mapper/gentoo-root /mnt/root || rescue

-rebundle your initramfs, copy to /boot

FizzyWidget · Post by **FizzyWidget** » Fri Nov 30, 2012 1:14 pm

still getting an error about /newroot

see img

http://imageshack.us/photo/my-images/29/img0002ko.jpg/

I am guessing it is related to this line

Code: Select all

exec /bin/busybox switch_root /newroot /sbin/init ${CMDLINE}

should this be changed to /mnt/root or /dev/mapprer/gentoo-root or just / ?

Do i need to add newroot to the init?

So close, and yet so far :/

cach0rr0 · Post by **cach0rr0** » Fri Nov 30, 2012 7:44 pm

Dark Foo wrote:still getting an error about /newroot

see img

http://imageshack.us/photo/my-images/29/img0002ko.jpg/

I am guessing it is related to this line
Code: Select all
exec /bin/busybox switch_root /newroot /sbin/init ${CMDLINE}
should this be changed to /mnt/root or /dev/mapprer/gentoo-root or just / ?

Do i need to add newroot to the init?

So close, and yet so far :/

can you post your initramfs structure?
just cd to the root of your initramfs, and:

Code: Select all

find . |xargs file

unless you went to the root of your initramfs directory, and did a 'mkdir newroot', then there is no directory named /newroot in your initramfs, and ergo, nowhere to mount your now raid'd and decrypted rootfs

FizzyWidget · Post by **FizzyWidget** » Fri Nov 30, 2012 8:21 pm

got it working

although i did notice a few errors on boot up but they went by so fast i couldnt see them, so i will ssh in and look at some logs

Code: Select all

* Setting up dm-crypt mappings ...
 *   gentoo-swap using: -c aes -h sha1 -d /dev/urandom create gentoo-swap /dev/md2 ...
 * failure running cryptsetup                                                                                                                                     [ !! ]
 *   gentoo-home using:   create gentoo-home /dev/md3 ...
 * failure running cryptsetup                                                                                                                                     [ !! ]
 * Failed to setup dm-crypt devices                                                                                                                               [ !! ]
 * ERROR: dmcrypt failed to start

dmcrypt

Code: Select all

swap=gentoo-swap
source='/dev/md2'

It says that because the raid is a luks drive that it aborts the creation of swap

cach0rr0 · Post by **cach0rr0** » Fri Nov 30, 2012 11:09 pm

Dark Foo wrote: It says that because the raid is a luks drive that it aborts the creation of swap

since your swap volume is luks, it should be handled just as any other disk as far as dmcrypt is concerned at least, instead of listed specifically as swap

so instead of

Code: Select all

swap=crypt-swap

you use 'target', like so:

Code: Select all

target=crypt-swap
source='/dev/md2'

or if it uses a keyfile

Code: Select all

target=crypt-swap
source='/dev/md2'
key='/path/to/swap.key'

FizzyWidget · Post by **FizzyWidget** » Fri Nov 30, 2012 11:13 pm

ah so if i change it to

traget=gentoo-swap
source='/dev/md2'

it should be good to go? if so I will look at adding a keyfile to it once it boots without error

cach0rr0 · Post by **cach0rr0** » Fri Nov 30, 2012 11:22 pm

Dark Foo wrote:ah so if i change it to

traget=gentoo-swap
source='/dev/md2'

it should be good to go? if so I will look at adding a keyfile to it once it boots without error

correct.
ASSUMING openrc is clever enough to make sure md2 is assembled and whatnot *before* trying to run dmcrypt

if it isn't, it's easy enough to sort out, just have to tweak the init scripts a bit. cross that bridge when we come to it

FizzyWidget · Post by **FizzyWidget** » Sat Dec 01, 2012 12:21 pm

all working now, have added a key to it and it boots fine, just need to put nuvou or how ever you spell it, so the nvidia card switches over to a less noisy state