Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
fdisk warning: error 22 after mdadm
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo on Sparc
View previous topic :: View next topic  
Author Message
Metalkhan
n00b
n00b


Joined: 08 Jan 2008
Posts: 6
Location: Hellas

PostPosted: Mon Jan 28, 2008 1:58 pm    Post subject: fdisk warning: error 22 after mdadm Reply with quote

Please help me...
I have installed gentoo on a Sun Fire V880, with QLA2200 (FCAL) and 6 identical SCSI hard disks. I'm running 2.6.17-gentoo-r9, and I have enabled all raid and multipathing in the kernel.

Code:

SFV880 ~ # cat /usr/src/linux/.config | grep SYM53C8XX
CONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
CONFIG_SCSI_SYM53C8XX_MMIO=y

SFV880 ~ # cat /usr/src/linux/.config | grep QLA
CONFIG_SCSI_QLA_FC=y
CONFIG_SCSI_QLA2XXX_EMBEDDED_FIRMWARE=y
CONFIG_SCSI_QLA21XX=y
CONFIG_SCSI_QLA22XX=y
CONFIG_SCSI_QLA2300=y
CONFIG_SCSI_QLA2322=y
CONFIG_SCSI_QLA24XX=y

SFV880 ~ # cat /usr/src/linux/.config | grep RAID
CONFIG_RAID_ATTRS=y
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
# Multi-device support (RAID and LVM)
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=y
CONFIG_MD_RAID5=y
CONFIG_MD_RAID5_RESHAPE=y
CONFIG_MD_RAID6=y

SFV880 ~ # cat /usr/src/linux/.config | grep MULTIPATH
CONFIG_MD_MULTIPATH=y
CONFIG_DM_MULTIPATH=y
CONFIG_DM_MULTIPATH_EMC=y


When I do fdisk -l it shows 12 disks, instead of 6. This is correct, because of the FCAL, it sees each of the 6 SCSI hard disks twice... once for each one of the 2 paths. I compared the output from blkid and found out that the 6 first hard disks match the UUIDs of the second 6 hard disks, respectively. I, identically partitioned all 6 hard disks using fdisk to look like this:

Code:
Disk /dev/sdf (Sun disk label): 27 heads, 107 sectors, 24620 cylinders
Units = cylinders of 2889 * 512 bytes

   Device Flag    Start       End    Blocks   Id  System
/dev/sdf1             0      1385   2000632+  fd  Linux raid autodetect
/dev/sdf2          1385      2770   2000632+  fd  Linux raid autodetect
/dev/sdf3             0     24620  35563590    5  Whole disk
/dev/sdf4          2770      4155   2000632+  fd  Linux raid autodetect
/dev/sdf5          4155      5540   2000632+  fd  Linux raid autodetect
/dev/sdf6          5540     24620  27561060   fd  Linux raid autodetect


I then looked into /dev, and saw all 12 disks with their partitions (i.e sda, sda1, sda2...), but no md device. I run mdadm -Es and it found 6 multipath arrays of 2 hard disks each, as it should (I checked each one with mdadm -D). I started the arrays using mdadm -As and looked into /dev again. I noticed, it created an /md directory and the devices md0 to md5, but deleted all the partitions of all the disks except the one I have the system on. Now, If I try to partition any of the md devices with fdisk, after I'm done and pressed w (write) I get the following error:

Code:
WARNING: Re-reading the partition table failed with error 22: Invalid arguement.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.


After reboot, nothing changes and I cannot create a filesystem on the partitions, since they don't appear in /dev.

What I want to do is, create all 6 multipath arrays (md0 to md5) and then a RAID 5 array (md6) with these 6 multipath devices. Also, the md6 (RAID 5) should have the above partition table (md6p1 to md6p6) and of course mount my system on it ;P My machine is a SPARC though and multipath tools won't do, so the only thing I have to do is mdadm. Can anyone help?
_________________
Kind Regards,
Michael
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 33044
Location: 56N 3W

PostPosted: Mon Jan 28, 2008 7:50 pm    Post subject: Reply with quote

Metalkhan,

You do not partition a kernel raid set. Your md raid set is composed of the underlying paritions.
You can if you wish nest raid sets, so that /dev/md2 is formed from /dev/md1 and /dev/md0 (and others as required)

You can use LVM on top of raid but I've never tried it on SPARC
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Metalkhan
n00b
n00b


Joined: 08 Jan 2008
Posts: 6
Location: Hellas

PostPosted: Tue Jan 29, 2008 9:15 am    Post subject: Reply with quote

NeddySeagoon wrote:
Metalkhan,

You do not partition a kernel raid set. Your md raid set is composed of the underlying paritions.
You can if you wish nest raid sets, so that /dev/md2 is formed from /dev/md1 and /dev/md0 (and others as required)

You can use LVM on top of raid but I've never tried it on SPARC


Neddy, I'm not sure I clearly understand what you mean by saying "kernel raid set" and that the md raid set is composed of the underlying partitions... Can you please elaborate a bit more??? Thanks in advance for your time.
_________________
Kind Regards,
Michael
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 33044
Location: 56N 3W

PostPosted: Tue Jan 29, 2008 9:28 pm    Post subject: Reply with quote

Metalkhan,

Linux supports three types of raid. BIOS software raid (also called fakeraid) , real hardware raid and kernel raid.
When you use the first two, your whole drives are donated to the raid sets and made to appear as a single device which you parition and make file systems on in the normal way. The physical reality of the raid set being composed of a number of drives is hidden from the kernel. With fakeraid, the kernel may be able to see the underying drives but you must not use them.

With kernel raid, the situation is different. You partition your drives and using mdadm donate the partitions to a raid set /dev/mdX.
This means that the same drives can support different raid levels on different partitions. An example may help.
Assume 3 drives with 3 partitions each /dev/sd[abc][123]
mdadm can make a raid 5 set from /dev/sd[abc]3, a raid0 set from /dev/sd[abc]2 a raid1 set from /dev/sd[bc]1 and leave /dev/sda1 unraided. You can mix and match as you like. You can also make use mdadm you make raid sets on top of raid sets ... feed it the /dev/mdX nodes as if they were hard drive partitions.
However, having made your kernel raid set(s), you cannot partition it(them) further, at least, not with fdisk. You can use Logical Volume Manager on top of raid, which achieves the same operational effect.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
chiguire
n00b
n00b


Joined: 20 Jan 2003
Posts: 29

PostPosted: Mon Feb 11, 2008 4:54 pm    Post subject: Reply with quote

NeddySeagoon wrote:

...
You can use LVM on top of raid but I've never tried it on SPARC


FWIW, my u2 runs lvm2 on 2 separate raid5 arrays (disks in a 711 box), exporting lvs out over nfs, working 24x7 quite well, thanks very much :D using a 2.6.22-r8 kernel, sys-fs/mdadm-2.6.2 and sys-fs/lvm2-2.02.33.

Not the fastest setup around by any means, but respectably solid and well behaved.
Back to top
View user's profile Send private message
Metalkhan
n00b
n00b


Joined: 08 Jan 2008
Posts: 6
Location: Hellas

PostPosted: Sat Feb 16, 2008 8:02 am    Post subject: Reply with quote

Neddy thanks for your time and info. I followed your instructions and ignored the later 6 hard disks.
I partitioned my first 6 disks as I show below:
Code:
/dev/sda1             0      1385   2000632+  fd  Linux raid autodetect
/dev/sda2  u       1385      2770   2000632+  fd  Linux raid autodetect
/dev/sda3             0     24620  35563590    5  Whole disk
/dev/sda4          2770     24620  31562325   fd  Linux raid autodetect


Then, I created a raid 1 set (md1) using the 1st partition of all 6 disks for ./ (root), a raid 5 set (md2) using the 2nd partition for swap and another raid 5 set (md4) using the 4th partition for everything else.

I activated LVM2 on the md4 and partitioned it for ./usr ./var ./home.

I installed gentoo-sources-2.6.17-r9 and everything seems ok except that when I boot, silo cannot find silo.conf. I havn't a separate partition for boot. I have run silo both inside and outside chroot but still I get the same results. I have copied silo on each sdisk and run silo for each one of them separately as says here, but when I boot, again silo cannot find silo.conf. If I type the path for the boot image and root=/dev/md1 it boots normally and runs fine but it's annoying.

What have I done wrong?

Maybe this post should go there https://forums.gentoo.org/viewtopic-t-578017-highlight-silo+conf+unknown+ext2+error.html, though noone has posted a solution, that's why I posted here.
_________________
Kind Regards,
Michael
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on Sparc All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum