How to do a gentoo install on a software RAID

Message

Obz · Post by **Obz** » Tue Apr 29, 2003 3:58 am

just some possible errata/corrections, re the original post:

RAID 0: 2 or more hard drives are combined into one big volume. The final volume size is the sum of all the drives. When data is written to the RAID drive, the data is written to all drives in the RAID 0 . This means that drive reads and writes are very fast, but if 1 drive dies you lose all your data.

While I understand that the poster may have meant something along the lines of, "the data is written across all drives" it's possible that the definition may be prone to misinterpretation such as,

"give me some data" -> "now write that data to each(all) of the drives"

which is clearly not RAID0.

Just thought someone might like to update that as those new to RAID (like I was when I first read this topic) could easily be confused.

Thanks,
Mike.

golemB · Post by **golemB** » Fri May 02, 2003 6:17 am

Alright, so between this helpful post and a few other webpages (see below), I was able to get these drives recognized by Linux as /dev/md0, and ``fdisk -l /dev/md0'' even was happy to report the existence of the vfat and ntfs partitions on the array! Woohoo Linux!

Now, maybe because I'm doing this from a Gentoo boot CD and not from a functional bootable hard drive, even after I do a mkraid to achieve this, there is no such device as /dev/md0p1 and /dev/md0p2, which are mysteriously what fdisk reports are the locations of my old, old partitions. So I can't mount these ``partitions'' within the /dev/md0 ``drive''. Unlike what is recommended in this post, my initial (pre-Linux) setup had both raw drives acting as a single drive in RAID 0. So my raidtab is set accordingly. Now how do I get mount to see these partitions md0p1 and md0p2? Do I have to reboot and have the kernel see the ``drive'' before it will accept this? In other words, must I get a functional Linux kernel running on a separate boot drive before I can start reading data off my array?

Many thanks,
golemB

References:

Another great general Linux RAID intro / howto:
http://www.linuxplanet.com/linuxplanet/ ... ls/4349/1/

Manpage for /etc/raidtab:
http://www.linuxvalley.it/encyclopedia/ ... dtab.5.php

bryon · Post by **bryon** » Sun May 18, 2003 3:27 am

I am trying to make a raid 1 setup with a spare. All 40GB drives on a P2(350). The problem that I am having is that when I do mkraid /dev/md* i get

cdimage root # mkraid /dev/md*
detected error on line 1:
Code:
mkraid: aborted, see the syslog and /proc/mdstat for potential clues.
cdimage root #

and /proc/mdstat gives me

Personalities :
read_ahead not set
unused devices: <none>

i am pretty stumped here, anny sugestions?

Obz · Post by **Obz** » Sun May 18, 2003 4:22 am

we'll need to have a look at your raidtab, so if you could post that please it might help, thanks.

mike.

bryon · Post by **bryon** » Sun May 18, 2003 4:58 am

My /etc/raidtab is as follows

# / (RAID 1 with hot-spare)
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
nr-spare-disks 1
chunk-size 32
persistent-superblock 1
device /dev/hda3
raid-disk 0
device /dev/hdd3
raid-disk 1
device /dev/hdb1
spare-disk 0

I have hda and hdb on ide one and hdd on the second ide. What I want it to do is have hda and hdd mirrors of each other(keep them of diffrent channels so if one channel fails) and hdb the hot spare.
I just figured out that i had some extra code in the /etc/raidtab but can you make sure that the above is what I am trying to do.

I seam to have a new problem now

device /dev/md199 is not described in config file
handling MD device /dev/md2
analyzing super-block
disk 0: /dev/hda3, 38472840kB, raid superblock at 38472768kB
disk 1: /dev/hdd3, 39425400kB, raid superblock at 39425280kB
disk 2: /dev/hdb1, 98248kB, raid superblock at 98176kB
/dev/hdb1 appears to be already part of a raid array -- use -f to
force the destruction of the old superblock
mkraid: aborted, see the syslog and /proc/mdstat for potential clues.

mkraid: aborted is not supos to happen right?

Obz · Post by **Obz** » Sun May 18, 2003 5:19 am

your raidtab configuration looks ok, that's the correct configuration for the spare configuration you're after. i find the line

device /dev/md199 is not described in config file

rather odd, but it doesnt seem to be fatal.

the real error is in:

/dev/hdb1 appears to be already part of a raid array -- use -f to
force the destruction of the old superblock

implying that the hard drive was previously part of a raid array.

try running

Code: Select all

mkraid -f /dev/md2

which should force the overwriting of the previous superblock.

bryon · Post by **bryon** » Mon May 19, 2003 2:15 am

when i do mkraid -f /dev/md* I get this big warning.

cdimage root # mkraid -f /dev/md*
--force and the new RAID 0.90 hot-add/hot-remove functionality should be
used with extreme care! If /etc/raidtab is not in sync with the real array
configuration, then a --force will DESTROY ALL YOUR DATA. It's especially
dangerous to use -f if the array is in degraded mode.

PLEASE dont mention the --really-force flag in any email, documentation or
HOWTO, just suggest the --force flag instead. Thus everybody will read
this warning at least once It really sucks to LOSE DATA. If you are
confident that everything will go ok then you can use the --really-force
flag. Also, if you are unsure what this is all about, dont hesitate to
ask questions on linux-raid@vger.rutgers.edu

so then i run plain mkraid /dev/md* and get the same error.
How should I get around it?

Obz · Post by **Obz** » Mon May 19, 2003 3:26 am

basically the warning is the raidtools covering their arses in the best way they can, _just_in_case_ something goes wrong. there are two reasons why you're getting all this.

it's possible that hdb1 was previously part of another raid array, im not sure about this because i dont know the history of your hard drives. if this is the case, then you clearly dont intend to use that previous array anymore, and so there is no reason why you cant overwrite the superblock and get on with it.

the second possibility is that earlier in this process, perhaps when you were changing your raidtab or something like that, you accidentally wrote the superblock on that drive without writing the other ones (it's easy to do things like this if your config is just slightly off etc, ive done it before). in this case, again, there's no reason why you cant overwrite the superblock now you have the correct configuration.

the bottom line is, if _you_ are absolutely sure that you want to use hdb1 in _this_ raid array, then give it the --really-force flag and it will go and create the array for you. the bit about your config being in sync with your raid array is fine - the raidtab you have setup _is_ correct for the configuration you intend.

Code: Select all

mkraid --really-force /dev/md2

should be what you're after. i'm trusting that since you want to create a new array on these disks/partitions you dont have any vital data left on them, because you might want to move that somewhere permanent first :)

mike.

bryon · Post by **bryon** » Mon May 19, 2003 3:56 am

Yes these where all 40GB disks that are having a new install on and does not matter what relly happens to them since I am doing a new install. Tahnks for your help. Hopefully the rest of the install will go smothly.

Obz · Post by **Obz** » Mon May 19, 2003 4:12 am

heh, not a problem. good luck with the rest of it.

mike.

Forse · Post by **Forse** » Mon May 19, 2003 2:21 pm

I was just wondering that how much does software raid stress CPU. I have Pentium III 500Mhz with ~1gig ram. I was wondering will it be capable of softwareRAID 5? It's server...and I would use 3 scsi disk. Any ideas, comments?

puke · Post by **puke** » Thu May 22, 2003 10:36 am

I was just wondering that how much does software raid stress CPU. I have Pentium III 500Mhz with ~1gig ram. I was wondering will it be capable of softwareRAID 5? It's server...and I would use 3 scsi disk. Any ideas, comments?

You should be fine, providing you are not doing any intensive read/writes.

I have moved to software RAID 1 on several Gentoo boxen and have never noticed any performance decrease. These machines are ~800MHz but only 0.5GB RAM. But none are doing any excessive disk I/O.

Software RAID seems to be really solid. Always make sure you have offline copies of docs, fstab and raidtab, in the event of a crash. Test rebuilding your array etc.

Forse · Post by **Forse** » Thu May 22, 2003 2:08 pm

I don't like using fdisk (too geeky), what should I do if I want to use cfdisk? I mean that is something important to know if I am planning to use cfdisk? In linuxdoc.org guide to SoftwareRAID:

Code: Select all

The partition-types of the devices used in the RAID must be set to 0xFD (use fdisk and set the type to ``fd'')

How to do that with cfdisk?

thnx in advance

Forse · Post by **Forse** » Thu May 22, 2003 2:52 pm

And, since I want xfs on the / and /home RAIDs
Code: Select all
#mkfs.xfs -d agcount=3 -l size=32m /dev/md2 
#mkfs.xfs -d agcount=3 -l size=32m /dev/md3
The parameters added to the mkfs.xfs command come from the suggestions made in orginal x86 install guide. Both my / and /home partitions are about 9 GB, and XFS likes at least one allocation group per 4 GB. Thus I used an agcount of 3.

I am planning to have 200gig RAID 0 with xfs, what parameters would u recommend for mkfs.xfs and for chunksize in raidtab.

thnx a lot

MehdiYM · Post by **MehdiYM** » Sat May 31, 2003 6:38 am

Hello,
I have a KT3 Ultra mother board with a ATA133 RAID.
I have bought a maxtor HDD 40Go ATA133 and I have connected it to the first connector of the RAID controller.
Then I have activated the RAID in the BIOS.
I have activated the modules md, raid0 and raid1 of my kernel.
I have booted my linux box and my new HDD has been detected as /dev/hde.
So I haven't made anything of your HOWTO and it works.
Why ?

golemB · Post by **golemB** » Sat May 31, 2003 3:28 pm

MehdiYM wrote: I have a KT3 Ultra mother board with a ATA133 RAID.
I have bought a maxtor HDD 40Go ATA133 and I have connected it to the first connector of the RAID controller.
Then I have activated the RAID in the BIOS.

First of all, RAID requires at least two hard drives, and they should be the same brand, model, and size for best results. If you don't understand why at this time, then you probably shouldn't try to use RAID.

golemB · Post by **golemB** » Tue Jun 03, 2003 9:34 am

Quick update - I put in a plain hard drive and got a reasonable Gentoo system up and running (multiboot w/ Win98 for kicks). Then I tried again with the RAID setup in another vain attempt to resurrect my old data. Yet again, the strange "/dev/md0p1" thing is still reported by fdisk but still cannot be mounted. I think I shall have to simply wipe out the disks and start anew.

puddpunk · Post by **puddpunk** » Sun Jun 08, 2003 6:11 am

MehdiYM wrote:Hello,
I have a KT3 Ultra mother board with a ATA133 RAID.
I have bought a maxtor HDD 40Go ATA133 and I have connected it to the first connector of the RAID controller.
Then I have activated the RAID in the BIOS.
I have activated the modules md, raid0 and raid1 of my kernel.
I have booted my linux box and my new HDD has been detected as /dev/hde.
So I haven't made anything of your HOWTO and it works.
Why ?

Because that's hardware RAID isn't it. The hardware is doing all the work of RAID-ing your drives together (the onboard controller on your motherboard). What is described in this HOWTO is software RAID, where the kernel does all the work of RAID-ing your hard drives together.

MehdiYM · Post by **MehdiYM** » Sun Jun 08, 2003 8:03 am

thx puddpunk and golemB, is there a way to put lilo (or grub) on a HDD that is connected on my raid controller ?
My aim is to have a dual boot with linux & winxp but without using the NTloader.
As winxp use the MBR of my first HDD (IDE) I though I can install lilo on the MBR of my second HDD (RAID), is it possible ?

paleck · Post by **paleck** » Mon Jun 23, 2003 12:40 am

I just wanted to say that those instructions were perfect for my set-up here. Thanks for taking the time to put them up.

xunil · Post by **xunil** » Tue Jun 24, 2003 8:23 pm

Some folks who don't mind losing their system in case of disk failure or who buy more reliable disks than retail or OEM IDE drives

might be interested in using RAID-0 on / for maximum performance. This doesn't "just work" like it does w/ RAID-1; instead you must provide some information to the kernel on the boot command line to jump-start the RAID-0 array so the kernel can mount it. At the end of your kernel line in your /boot/grub/grub.conf file, append something like the following:

Code: Select all

md=0,/dev/hda1,/dev/hdc1

Let me explain this so you can write your own for your personal configuration: 0 is the md device which you'll be jump-starting (in this case, 0). Following your md device number is a comma-seperated list of the devices which make up that md device (in this case, /dev/hda1 and /dev/hdc1). So, let's say your md device which you want to mount as / is 2 which corresponds to /dev/md2, and /dev/md2 is composed of /dev/hda3, /dev/hdc2, and /dev/hde1. Here's what you'd append to your kernel line in /boot/grub/grub.conf:

Code: Select all

md=2,/dev/hda3,/dev/hdc2,/dev/hde1

One caveat: these instructions only work if your md device uses a persistent superblock. If not (there's no reason not to, BTW), read /usr/src/linux/Documentation/md.txt.

greg32 · Post by **greg32** » Thu Jun 26, 2003 8:36 pm

can you guys have a look at my problem (link posted below) cause I am having real problems getting software raid0 to work on my system.

http://forums.gentoo.org/viewtopic.php?t=62575
thanks

wrex · Post by **wrex** » Fri Jun 27, 2003 4:10 am

Just curious why no-one seems to be using mdadm instead of raidtools?

Seems a much cleaner interface to me (the 50 or so lines in /etc/init.d/checkfs are reduced to one line: "mdadm --assemble --scan").

There is a nice O'Reilly article on mdadm.

(I submitted bug 2437 to request that checkfs support mdadm and to change the startup order of raid and lvm -- raid should be started first, shouldn't it?!!)

wrex · Post by **wrex** » Fri Jun 27, 2003 4:29 am

chatwood2 wrote:/boot and / will be a RAID 1 (mirror)

Since grub needs a real physical disk partition to bootstrap the OS, I don't see the value of mirroring /boot (especially since gentoo is wise enough to leave /boot unmounted by default).

My preference is to do a "poor man's RAID-1" for /boot. I make /dev/hda1 and /dev/hdb1 equal sized and periodically copy /dev/hda1 to /dev/hdb1 by hand. The backup command is simply

Code: Select all

dd if=/dev/hda1 of=/dev/hdb1 bs=8192b

In my case, "periodically" means every time I make a change to /boot (but not until I've successfully booted with the change!). The copy only takes a few seconds for a normal sized (100MB or less) /boot.

Not making the copy until you've verified the change is okay (by booting) is an important point (and actually a very good argument for NOT putting /boot on a RAID1 mirror -- human error in writing to /boot is far more likely than a disk failure [at least in my case!].

[Auspex (RIP) explicitly did NOT mirror the OS drive in their NFS file servers for exactly this reason (and manually "cloned" the OS drive after upgrades instead).]

I've even tinkered with the idea of putting a script that runs at the very end of the default runlevel that looks something like

Code: Select all

(sleep 3600; dd if=/dev/hda1 of=/dev/hdb1) &

I don't build a new kernel all that often though, so I'm content to just manually copy to hdb after booting a new kernel (or bootsplash image or any other change to /boot).

jerome187 · Post by **jerome187** » Mon Jul 14, 2003 7:48 am

i'm having trouble with step #6, the mkraid thingie.

Code: Select all

cdimage root # mkraid /dev/md0
cannot determine md version: No such file or directory

heres my /etc/raidtab/

Code: Select all

# / (RAID 0)
raiddev                 /dev/md0
raid-level              0
nr-raid-disks           4
chunk-size              32
persistent-superblock   1
device                  /dev/sda1
raid-disk               0
device                  /dev/sdb1
raid-disk               1
device                  /dev/sdc1
raid-disk               2
device                  /dev/sdd1
raid-disk               3


# swap (RAID 0)
raiddev                 /dev/md1
raid-level              0
nr-raid-disks           4
chunk-size              32
persistent-superblock   1
device                  /dev/sda2
raid-disk               0
device                  /dev/sdb2
raid-disk               1
device                  /dev/sdc2
raid-disk               2
device                  /dev/sdd2
raid-disk               3

i have 4 SCSI disks (all same size make model even partions are the same sizes). I only want a / and swap partions. whats wrong?

How to do a gentoo install on a software RAID

partial success?

mkraid /dev/md* problems

my raid tab

mkraid -f /dev/md*

40gb dedicated to the array

CPU cycles

Problem still unsolved.

Raid

Using RAID-0 for /

why raidtools/mkraid instead of mdadm?

manually "cloning" /boot instead of mirroring it