| View previous topic :: View next topic |
| Author |
Message |
Metalkhan
n00b
Joined: 08 Jan 2008
Posts: 6
Location: Hellas
|
 Posted: Mon Jan 28, 2008 1:58 pm Post subject: fdisk warning: error 22 after mdadm |
 |
|
Please help me...
I have installed gentoo on a Sun Fire V880, with QLA2200 (FCAL) and 6 identical SCSI hard disks. I'm running 2.6.17-gentoo-r9, and I have enabled all raid and multipathing in the kernel.
| Code: |
SFV880 ~ # cat /usr/src/linux/.config | grep SYM53C8XX
CONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
CONFIG_SCSI_SYM53C8XX_MMIO=y
SFV880 ~ # cat /usr/src/linux/.config | grep QLA
CONFIG_SCSI_QLA_FC=y
CONFIG_SCSI_QLA2XXX_EMBEDDED_FIRMWARE=y
CONFIG_SCSI_QLA21XX=y
CONFIG_SCSI_QLA22XX=y
CONFIG_SCSI_QLA2300=y
CONFIG_SCSI_QLA2322=y
CONFIG_SCSI_QLA24XX=y
SFV880 ~ # cat /usr/src/linux/.config | grep RAID
CONFIG_RAID_ATTRS=y
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
# Multi-device support (RAID and LVM)
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=y
CONFIG_MD_RAID5=y
CONFIG_MD_RAID5_RESHAPE=y
CONFIG_MD_RAID6=y
SFV880 ~ # cat /usr/src/linux/.config | grep MULTIPATH
CONFIG_MD_MULTIPATH=y
CONFIG_DM_MULTIPATH=y
CONFIG_DM_MULTIPATH_EMC=y |
When I do fdisk -l it shows 12 disks, instead of 6. This is correct, because of the FCAL, it sees each of the 6 SCSI hard disks twice... once for each one of the 2 paths. I compared the output from blkid and found out that the 6 first hard disks match the UUIDs of the second 6 hard disks, respectively. I, identically partitioned all 6 hard disks using fdisk to look like this:
| Code: | Disk /dev/sdf (Sun disk label): 27 heads, 107 sectors, 24620 cylinders
Units = cylinders of 2889 * 512 bytes
Device Flag Start End Blocks Id System
/dev/sdf1 0 1385 2000632+ fd Linux raid autodetect
/dev/sdf2 1385 2770 2000632+ fd Linux raid autodetect
/dev/sdf3 0 24620 35563590 5 Whole disk
/dev/sdf4 2770 4155 2000632+ fd Linux raid autodetect
/dev/sdf5 4155 5540 2000632+ fd Linux raid autodetect
/dev/sdf6 5540 24620 27561060 fd Linux raid autodetect |
I then looked into /dev, and saw all 12 disks with their partitions (i.e sda, sda1, sda2...), but no md device. I run mdadm -Es and it found 6 multipath arrays of 2 hard disks each, as it should (I checked each one with mdadm -D). I started the arrays using mdadm -As and looked into /dev again. I noticed, it created an /md directory and the devices md0 to md5, but deleted all the partitions of all the disks except the one I have the system on. Now, If I try to partition any of the md devices with fdisk, after I'm done and pressed w (write) I get the following error:
| Code: | WARNING: Re-reading the partition table failed with error 22: Invalid arguement.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks. |
After reboot, nothing changes and I cannot create a filesystem on the partitions, since they don't appear in /dev.
What I want to do is, create all 6 multipath arrays (md0 to md5) and then a RAID 5 array (md6) with these 6 multipath devices. Also, the md6 (RAID 5) should have the above partition table (md6p1 to md6p6) and of course mount my system on it ;P My machine is a SPARC though and multipath tools won't do, so the only thing I have to do is mdadm. Can anyone help?
_________________
Kind Regards,
Michael |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54216
Location: 56N 3W
|
| Posted: Mon Jan 28, 2008 7:50 pm Post subject: |
|
|
Metalkhan,
You do not partition a kernel raid set. Your md raid set is composed of the underlying paritions.
You can if you wish nest raid sets, so that /dev/md2 is formed from /dev/md1 and /dev/md0 (and others as required)
You can use LVM on top of raid but I've never tried it on SPARC
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Metalkhan
n00b
Joined: 08 Jan 2008
Posts: 6
Location: Hellas
|
| Posted: Tue Jan 29, 2008 9:15 am Post subject: |
|
|
| NeddySeagoon wrote: | Metalkhan,
You do not partition a kernel raid set. Your md raid set is composed of the underlying paritions.
You can if you wish nest raid sets, so that /dev/md2 is formed from /dev/md1 and /dev/md0 (and others as required)
You can use LVM on top of raid but I've never tried it on SPARC |
Neddy, I'm not sure I clearly understand what you mean by saying "kernel raid set" and that the md raid set is composed of the underlying partitions... Can you please elaborate a bit more??? Thanks in advance for your time.
_________________
Kind Regards,
Michael |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54216
Location: 56N 3W
|
| Posted: Tue Jan 29, 2008 9:28 pm Post subject: |
|
|
Metalkhan,
Linux supports three types of raid. BIOS software raid (also called fakeraid) , real hardware raid and kernel raid.
When you use the first two, your whole drives are donated to the raid sets and made to appear as a single device which you parition and make file systems on in the normal way. The physical reality of the raid set being composed of a number of drives is hidden from the kernel. With fakeraid, the kernel may be able to see the underying drives but you must not use them.
With kernel raid, the situation is different. You partition your drives and using mdadm donate the partitions to a raid set /dev/mdX.
This means that the same drives can support different raid levels on different partitions. An example may help.
Assume 3 drives with 3 partitions each /dev/sd[abc][123]
mdadm can make a raid 5 set from /dev/sd[abc]3, a raid0 set from /dev/sd[abc]2 a raid1 set from /dev/sd[bc]1 and leave /dev/sda1 unraided. You can mix and match as you like. You can also make use mdadm you make raid sets on top of raid sets ... feed it the /dev/mdX nodes as if they were hard drive partitions.
However, having made your kernel raid set(s), you cannot partition it(them) further, at least, not with fdisk. You can use Logical Volume Manager on top of raid, which achieves the same operational effect.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
chiguire
n00b
Joined: 20 Jan 2003
Posts: 29
|
| Posted: Mon Feb 11, 2008 4:54 pm Post subject: |
|
|
| NeddySeagoon wrote: |
...
You can use LVM on top of raid but I've never tried it on SPARC |
FWIW, my u2 runs lvm2 on 2 separate raid5 arrays (disks in a 711 box), exporting lvs out over nfs, working 24x7 quite well, thanks very much  using a 2.6.22-r8 kernel, sys-fs/mdadm-2.6.2 and sys-fs/lvm2-2.02.33.
Not the fastest setup around by any means, but respectably solid and well behaved. |
|
| Back to top |
|
|
Metalkhan
n00b
Joined: 08 Jan 2008
Posts: 6
Location: Hellas
|
| Posted: Sat Feb 16, 2008 8:02 am Post subject: |
|
|
Neddy thanks for your time and info. I followed your instructions and ignored the later 6 hard disks.
I partitioned my first 6 disks as I show below:
| Code: | /dev/sda1 0 1385 2000632+ fd Linux raid autodetect
/dev/sda2 u 1385 2770 2000632+ fd Linux raid autodetect
/dev/sda3 0 24620 35563590 5 Whole disk
/dev/sda4 2770 24620 31562325 fd Linux raid autodetect |
Then, I created a raid 1 set (md1) using the 1st partition of all 6 disks for ./ (root), a raid 5 set (md2) using the 2nd partition for swap and another raid 5 set (md4) using the 4th partition for everything else.
I activated LVM2 on the md4 and partitioned it for ./usr ./var ./home.
I installed gentoo-sources-2.6.17-r9 and everything seems ok except that when I boot, silo cannot find silo.conf. I havn't a separate partition for boot. I have run silo both inside and outside chroot but still I get the same results. I have copied silo on each sdisk and run silo for each one of them separately as says here, but when I boot, again silo cannot find silo.conf. If I type the path for the boot image and root=/dev/md1 it boots normally and runs fine but it's annoying.
What have I done wrong?
Maybe this post should go there https://forums.gentoo.org/viewtopic-t-578017-highlight-silo+conf+unknown+ext2+error.html, though noone has posted a solution, that's why I posted here.
_________________
Kind Regards,
Michael |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo on Sparc
