Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
raid5 && [UU_] && [U_U] [solved]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Mon Apr 05, 2004 10:57 am    Post subject: raid5 && [UU_] && [U_U] [solved] Reply with quote

i'm using a software raid5. my raidtab looks like this:
Code:
pts/1 cat /etc/raidtab
#swap (raid5)
raiddev /dev/md0
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
device  /dev/sdb2
raid-disk       0
device  /dev/sdc2
raid-disk       1
device /dev/sda2
raid-disk       2

#root (raid5)
raiddev /dev/md1
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
parity-algorithm left-symmetric
device  /dev/sdb3
raid-disk 0
device  /dev/sdc3
raid-disk 1
device  /dev/sda3
raid-disk 2

# /home (raid5)
raiddev /dev/md2
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
parity-algorithm left-symmetric
device /dev/sdb5
raid-disk 0
device /dev/sdc5
raid-disk 1
device /dev/sda5
raid-disk 2

# /free (raid5)
raiddev /dev/md3
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
parity-algorithm left-symmetric
device /dev/sdb6
raid-disk 0
device /dev/sdc6
raid-disk 1
device /dev/sda6
raid-disk 2

my fstab:
Code:
/dev/sdb1               /boot           ext2            noauto,noatime          1 2
/dev/md0                none         swap           sw                 0 0
/dev/md1                /               ext3            noatime         0 1
/dev/md2                /home       ext3            noatime         0 0
/dev/md3                /free         ext3            noatime         0 0

i somehow messed up my raid, as i'm having two drives down:
Code:
pts/1 cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid5] [multipath] [raid6]
md1 : active raid5 sdc3[1] sdb3[0]
      39085952 blocks level 5, 4k chunk, algorithm 2 [3/2] [UU_]

md2 : active raid5 sdc5[1] sdb5[0] sda5[2]
      97675008 blocks level 5, 4k chunk, algorithm 2 [3/3] [UUU]

md3 : active raid5 sdc6[1] sdb6[0] sda6[2]
      94124544 blocks level 5, 4k chunk, algorithm 2 [3/3] [UUU]

md0 : active raid5 sda2[2] sdb2[0]
      3148544 blocks level 5, 4k chunk, algorithm 0 [3/2] [U_U]

unused devices: <none>

md0 is my swap raid, which had one drive down from the beginning (i don'c care). but /dev/md1 is my root partition and i dont like one drive being down. i messed up the swap array and my /boot parition so i had to reboot with the gentoo cd. i struggled with stopping my raid devices so i could mount my /boot partition and since then one drive of my root partition is down.

any ideas on how to bring up the third drive again?


Last edited by BlinkEye on Sun Apr 11, 2004 2:02 pm; edited 1 time in total
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Mon Apr 05, 2004 10:43 pm    Post subject: Reply with quote

man raidtools
would be the best place to start
also
man mdadm


you would probablly need to start from the cd as your root drive is the problem

raidstop /dev/md1
raidhotadd -a /dev/sda3 /dev/md0
raidstart /dev/md0

might do something ?

see
Linux Software RAID HOWTO -- http://www.tldp.org/HOWTO/Software-RAID-HOWTO.html

MDADM MD device management package -- http://www.cse.unsw.edu.au/~neilb/source/mdadm/
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Mon Apr 05, 2004 11:36 pm    Post subject: Reply with quote

thanks. will give it a try asap.
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Thu Apr 08, 2004 10:21 pm    Post subject: Reply with quote

damn. it doesn't work. i get a
Code:
dev/hde3: not a MD device

any other suggestion?
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Fri Apr 09, 2004 12:00 pm    Post subject: Reply with quote

can we clarify which partition are used here. Your raidtab is
Code:

device  /dev/sdb3
raid-disk 0
device  /dev/sdc3
raid-disk 1
device  /dev/sda3
raid-disk 2

which is consistent with the error
Code:

md1 : active raid5 sdc3[1] sdb3[0]
      39085952 blocks level 5, 4k chunk, algorithm 2 [3/2] [UU_]

you now have an error
Code:

dev/hde3: not a MD device
 

If you are trying a different disk (hde3) then you need to change the RAIDTAB to match this (ie device /dev/hde3 raid-disk 2 ) Also ensure that the partition size is equal to or larger than the other two partitions in the array, and that the partition type is fd (linux raid autodetect) I assume you have done this with the other partitions.

If you are trying to remount your original sda3 then I would suggest checking the partition type and possibly clean formatting that partition only. This will remove the superblock etc which may encourage the array to rebuild. But try other possibilities first. to clean format use dd
[this is from memory so please check (its in the howto as I recall) dd if=/dev/zero of=/dev/sda3 ]
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Fri Apr 09, 2004 2:46 pm    Post subject: Reply with quote

thanks a lot for your answer. i'm sorry i haven't provided enough information. the issue is, that within the livecd environment two of my SATA Raid devices get mapped to hde and hdg (or something similar, only one SATA drive is referred to sda). this is why i must
Code:
raidhotadd /dev/hda3 /dev/md0


as you've seen my swap raid has one hd down too. i can experiment with that array. for raidhotadd-ing i don't need to boot into a cd-environment, but it doesn't work neither:
Code:
root@amd64 pts/3 swapoff /dev/md0

/home/blinkeye
root@amd64 pts/3 raidstop /dev/md0

/home/blinkeye
root@amd64 pts/3 raidhotadd -a /dev/sdc2 /dev/md0

/dev/sdc2: not an MD device!

/home/blinkeye
root@amd64 pts/3


my fdisk of sdc:
Code:
root@amd64 pts/3 fdisk /dev/sdc

The number of cylinders for this disk is set to 14593.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdc: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *           1          25      200781   83  Linux
/dev/sdc2              26         221     1574370   82  Linux swap
/dev/sdc3             222        2654    19543072+  fd  Linux raid autodetect
/dev/sdc4            2655       14593    95900017+   5  Extended
/dev/sdc5            2655        8734    48837568+  fd  Linux raid autodetect
/dev/sdc6            8735       14593    47062386   fd  Linux raid autodetect


i think your pretty sure on how to handle this problem. have you any further suggestion? i could dd this drive, but according to the error message i guess the superblock may not be a problem...

btw: you see now that my raid partitions are fd types, i checked that again (with the exception of the swap partition of course)
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Fri Apr 09, 2004 2:48 pm    Post subject: Reply with quote

slais-sysweb wrote:

If you are trying a different disk (hde3) then you need to change the RAIDTAB to match this (ie device /dev/hde3 raid-disk 2 ) Also ensure that the partition size is equal to or larger than the other two partitions in the array, and that the partition type is fd (linux raid autodetect) I assume you have done this with the other partitions.

there may be a point there nevertheless. i copied my original raidtab to the livecd-environment which of course isn't correct. but why isn't it working for the swap array?
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Fri Apr 09, 2004 6:36 pm    Post subject: Reply with quote

Well let's fix the swap array first.
As you are using RAID on that you need to make the partition type fd not linux swap. You then
Code:
mkswap /dev/md0

However there really isn't any need to set up raid for swap in this way. Well, there is an argument that RAID will give you some protection against the system going down if the swap drive fails, but swap is ephermeral so there is no long term data to recover after a reboot. If you use swap partitions (partition type 83 swap as you have) then just put these in fstab as separate partitions with pri=1 for each, linux itself will then use these in parralell for swap.
Code:

mkswap /dev/sda2
mkswap /dev/sdb2
mkswap /dev/sdc2

Code:

# /etc/fstab
/dev/sda2               none            swap            sw,pri=1                     0 0
/dev/sdb2               none            swap            sw,pri=1                    0 0     /dev/sdc2               none            swap            sw,pri=1                     0 0


As to the other problem, I think the answer is with getting the raidtab right. You must have a correct definition for the system you are working on in /etc/raidtab in order to use the raidtools. This means if you boot from the cd and the disk devs are different then you need to change (or copy an alternative version of) the raidtab. If you use the mdadm tools (which last time I needed them were a masked ebuild but I beleive are now unmasked) then once an array is created the raidtab is not used, but you will have to study the mdadm man pages and howto to get to grips with its method of refering to disks. It's a while since I last used these and as I am presently at home and I have just discovered that I can't ssh to my server at work I can't check how it all looks on a box I have set up.

I suppose the other thing to do is to verify that the disk/partition is itself ok
Let me know how you get on.
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Sat Apr 10, 2004 3:54 pm    Post subject: Reply with quote

ok, thanks for the guide on how to set up swapspace without using a raid array. i new it wasn't the best way, but it was that hard to set up my system i tried it that way first.

for my problem of drives being down: i tried with my swap partitions to get the third drive up, but it still doesn't work.
Code:
Disk /dev/sda: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          25      200781   83  Linux
/dev/sda2              26         221     1574370   fd  Linux raid autodetect
/dev/sda3             222        2654    19543072+  fd  Linux raid autodetect
/dev/sda4            2655       14593    95900017+   5  Extended
/dev/sda5            2655        8734    48837568+  fd  Linux raid autodetect
/dev/sda6            8735       14593    47062386   fd  Linux raid autodetect

all my 3 hd's look the same, but i still get the ugly
Code:
not a MD device
message. i don't know how to proceed...

well, i noticed a difference within my raidtab:
Code:
 cat /etc/raidtab
#swap (raid5)
raiddev /dev/md0
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
device  /dev/sdb2
raid-disk       0
device  /dev/sdc2
raid-disk       1
device /dev/sda2
raid-disk       2

#root (raid5)
raiddev /dev/md1
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
parity-algorithm left-symmetric
device  /dev/sdb3
raid-disk 0
device  /dev/sdc3
raid-disk 1
device  /dev/sda3
raid-disk 2

# /home (raid5)
raiddev /dev/md2
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
persistent-superblock 1
chunk-size 4
parity-algorithm left-symmetric
device /dev/sdb5
raid-disk 0
device /dev/sdc5
raid-disk 1
device /dev/sda5
raid-disk 2


could it be the missing
Code:
parity-algorithm left-symmetric

this is the line which is missing for my swap and / partition :?:
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Sat Apr 10, 2004 7:16 pm    Post subject: Reply with quote

Yes, I think parity algorithm may be the answer. Looking at the howto it looks to me as if withou that line you may not be getting raid 5

However, you seem to have that in the case of your md1 (/) array so that still leaves a problem.

I also notice your chunk size is a bit small 4 rather than the recomended 32, but perhaps you had best not change that on a sort of half working system, it may make things worse.
see http://www.tldp.org/HOWTO/Software-RAID-HOWTO-5.html#ss5.2
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Sat Apr 10, 2004 8:47 pm    Post subject: Reply with quote

slais-sysweb wrote:
I also notice your chunk size is a bit small 4 rather than the recomended 32, but perhaps you had best not change that on a sort of half working system, it may make things worse.
see http://www.tldp.org/HOWTO/Software-RAID-HOWTO-5.html#ss5.2

you may be right there, but according to that howto (which i used too):
Quote:
Thus, for large writes, you may see lower overhead by having fairly large chunks, whereas arrays that are primarily holding small files may benefit more from a smaller chunk size.

Chunk sizes must be specified for all RAID levels, including linear mode. However, the chunk-size does not make any difference for linear mode.

For optimal performance, you should experiment with the value, as well as with the block-size of the filesystem you put on the array.

so it may not be that bad as i don't use any large files like dvd rips or similar files.

back to my problem, i still get the same error. some part of my /var/log/messages which may help (unfortunately it doesn't help me):
Code:
Apr 10 22:30:10 amd64 md: linear personality registered as nr 1
Apr 10 22:30:10 amd64 md: raid0 personality registered as nr 2
Apr 10 22:30:10 amd64 md: raid1 personality registered as nr 3
Apr 10 22:30:10 amd64 md: raid5 personality registered as nr 4
Apr 10 22:30:10 amd64 raid5: measuring checksumming speed
Apr 10 22:30:10 amd64 generic_sse:  5564.000 MB/sec
Apr 10 22:30:10 amd64 raid5: using function: generic_sse (5564.000 MB/sec)
Apr 10 22:30:10 amd64 raid6: int64x1   1667 MB/s
Apr 10 22:30:10 amd64 raid6: int64x2   2492 MB/s
Apr 10 22:30:10 amd64 raid6: int64x4   2617 MB/s
Apr 10 22:30:10 amd64 raid6: int64x8   1750 MB/s
Apr 10 22:30:10 amd64 raid6: sse2x1     753 MB/s
Apr 10 22:30:10 amd64 raid6: sse2x2    1277 MB/s
Apr 10 22:30:10 amd64 raid6: sse2x4    1742 MB/s
Apr 10 22:30:10 amd64 raid6: using algorithm sse2x4 (1742 MB/s)
Apr 10 22:30:10 amd64 md: raid6 personality registered as nr 8
Apr 10 22:30:10 amd64 md: multipath personality registered as nr 7
Apr 10 22:30:10 amd64 md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
Apr 10 22:30:10 amd64 Advanced Linux Sound Architecture Driver Version 1.0.2c.
Apr 10 22:30:10 amd64 Trying to register duplicated ioctl32 handler 5420
Apr 10 22:30:10 amd64 Trying to register duplicated ioctl32 handler 5421
Apr 10 22:30:10 amd64 Trying to register duplicated ioctl32 handler 5422
Apr 10 22:30:10 amd64 devfs_mk_cdev: could not append to parent for sound/audio
Apr 10 22:30:10 amd64 ALSA device list:
Apr 10 22:30:10 amd64 #0: Sound Blaster Audigy (rev.3) at 0xa800, irq 18
Apr 10 22:30:10 amd64 oprofile: using NMI interrupt.
Apr 10 22:30:10 amd64 NET: Registered protocol family 2
Apr 10 22:30:10 amd64 IP: routing cache hash table of 8192 buckets, 128Kbytes
Apr 10 22:30:10 amd64 TCP: Hash tables configured (established 262144 bind 65536)
Apr 10 22:30:10 amd64 IPv4 over IPv4 tunneling driver
Apr 10 22:30:10 amd64 GRE over IPv4 tunneling driver
Apr 10 22:30:10 amd64 ip_conntrack version 2.1 (8192 buckets, 65536 max) - 448 bytes per conntrack
Apr 10 22:30:10 amd64 ip_tables: (C) 2000-2002 Netfilter core team
Apr 10 22:30:10 amd64 arp_tables: (C) 2002 David S. Miller
Apr 10 22:30:10 amd64 Initializing IPsec netlink socket
Apr 10 22:30:10 amd64 NET: Registered protocol family 1
Apr 10 22:30:10 amd64 NET: Registered protocol family 17
Apr 10 22:30:10 amd64 NET: Registered protocol family 15
Apr 10 22:30:10 amd64 ACPI: (supports S0 S1 S3 S4 S5)
Apr 10 22:30:10 amd64 md: Autodetecting RAID arrays.
Apr 10 22:30:10 amd64 md: autorun ...
Apr 10 22:30:10 amd64 md: considering sdc6 ...
Apr 10 22:30:10 amd64 md:  adding sdc6 ...
Apr 10 22:30:10 amd64 md: sdc5 has different UUID to sdc6
Apr 10 22:30:10 amd64 md: sdc3 has different UUID to sdc6
Apr 10 22:30:10 amd64 md: sdc2 has different UUID to sdc6
Apr 10 22:30:10 amd64 md:  adding sdb6 ...
Apr 10 22:30:10 amd64 md: sdb5 has different UUID to sdc6
Apr 10 22:30:10 amd64 md: sdb3 has different UUID to sdc6
Apr 10 22:30:10 amd64 md: sdb2 has different UUID to sdc6
Apr 10 22:30:10 amd64 md:  adding sda6 ...
Apr 10 22:30:10 amd64 md: sda5 has different UUID to sdc6
Apr 10 22:30:10 amd64 md: sda3 has different UUID to sdc6
Apr 10 22:30:10 amd64 md: sda2 has different UUID to sdc6
Apr 10 22:30:10 amd64 md: created mdX
Apr 10 22:30:10 amd64 md: bind<sda6>
Apr 10 22:30:10 amd64 md: bind<sdb6>
Apr 10 22:30:10 amd64 md: bind<sdc6>
Apr 10 22:30:10 amd64 md: running: <sdc6><sdb6><sda6>
Apr 10 22:30:10 amd64 raid5: device sdc6 operational as raid disk 1
Apr 10 22:30:10 amd64 raid5: device sdb6 operational as raid disk 0
Apr 10 22:30:10 amd64 raid5: device sda6 operational as raid disk 2
Apr 10 22:30:10 amd64 raid5: allocated 3202kB for md3
Apr 10 22:30:10 amd64 raid5: raid level 5 set md3 active with 3 out of 3 devices, algorithm 2
Apr 10 22:30:10 amd64 RAID5 conf printout:
Apr 10 22:30:10 amd64 --- rd:3 wd:3 fd:0
Apr 10 22:30:10 amd64 disk 0, o:1, dev:sdb6
Apr 10 22:30:10 amd64 disk 1, o:1, dev:sdc6
Apr 10 22:30:10 amd64 disk 2, o:1, dev:sda6
Apr 10 22:30:10 amd64 md: considering sdc5 ...
Apr 10 22:30:10 amd64 md:  adding sdc5 ...
Apr 10 22:30:10 amd64 md: sdc3 has different UUID to sdc5
Apr 10 22:30:10 amd64 md: sdc2 has different UUID to sdc5
Apr 10 22:30:10 amd64 md:  adding sdb5 ...
Apr 10 22:30:10 amd64 md: sdb3 has different UUID to sdc5
Apr 10 22:30:10 amd64 md: sdb2 has different UUID to sdc5
Apr 10 22:30:10 amd64 md:  adding sda5 ...
Apr 10 22:30:10 amd64 md: sda3 has different UUID to sdc5
Apr 10 22:30:10 amd64 md: sda2 has different UUID to sdc5
Apr 10 22:30:10 amd64 md: created mdX
Apr 10 22:30:10 amd64 md: bind<sda5>
Apr 10 22:30:10 amd64 md: bind<sdb5>
Apr 10 22:30:10 amd64 md: bind<sdc5>
Apr 10 22:30:10 amd64 md: running: <sdc5><sdb5><sda5>
Apr 10 22:30:10 amd64 raid5: device sdc5 operational as raid disk 1
Apr 10 22:30:10 amd64 raid5: device sdb5 operational as raid disk 0
Apr 10 22:30:10 amd64 raid5: device sda5 operational as raid disk 2
Apr 10 22:30:10 amd64 raid5: allocated 3202kB for md2
Apr 10 22:30:10 amd64 raid5: raid level 5 set md2 active with 3 out of 3 devices, algorithm 2
Apr 10 22:30:10 amd64 RAID5 conf printout:
Apr 10 22:30:10 amd64 --- rd:3 wd:3 fd:0
Apr 10 22:30:10 amd64 disk 0, o:1, dev:sdb5
Apr 10 22:30:10 amd64 disk 1, o:1, dev:sdc5
Apr 10 22:30:10 amd64 disk 2, o:1, dev:sda5
Apr 10 22:30:10 amd64 md: considering sdc3 ...
Apr 10 22:30:10 amd64 md:  adding sdc3 ...
Apr 10 22:30:10 amd64 md: sdc2 has different UUID to sdc3
Apr 10 22:30:10 amd64 md:  adding sdb3 ...
Apr 10 22:30:10 amd64 md: sdb2 has different UUID to sdc3
Apr 10 22:30:10 amd64 md:  adding sda3 ...
Apr 10 22:30:10 amd64 md: sda2 has different UUID to sdc3
Apr 10 22:30:10 amd64 md: created mdX
Apr 10 22:30:10 amd64 md: bind<sda3>
Apr 10 22:30:10 amd64 md: bind<sdb3>
Apr 10 22:30:10 amd64 md: bind<sdc3>
Apr 10 22:30:10 amd64 md: running: <sdc3><sdb3><sda3>
Apr 10 22:30:10 amd64 md: kicking non-fresh sda3 from array!
Apr 10 22:30:10 amd64 md: unbind<sda3>
Apr 10 22:30:10 amd64 md: export_rdev(sda3)
Apr 10 22:30:10 amd64 raid5: device sdc3 operational as raid disk 1
Apr 10 22:30:10 amd64 raid5: device sdb3 operational as raid disk 0
Apr 10 22:30:10 amd64 raid5: allocated 3202kB for md1
Apr 10 22:30:10 amd64 raid5: raid level 5 set md1 active with 2 out of 3 devices, algorithm 2
Apr 10 22:30:10 amd64 RAID5 conf printout:
Apr 10 22:30:10 amd64 --- rd:3 wd:2 fd:1
Apr 10 22:30:10 amd64 disk 0, o:1, dev:sdb3
Apr 10 22:30:10 amd64 disk 1, o:1, dev:sdc3
Apr 10 22:30:10 amd64 md: considering sdc2 ...
Apr 10 22:30:10 amd64 md:  adding sdc2 ...
Apr 10 22:30:10 amd64 md:  adding sdb2 ...
Apr 10 22:30:10 amd64 md:  adding sda2 ...
Apr 10 22:30:10 amd64 md: created md0
Apr 10 22:30:10 amd64 md: bind<sda2>
Apr 10 22:30:10 amd64 md: bind<sdb2>
Apr 10 22:30:10 amd64 md: bind<sdc2>
Apr 10 22:30:10 amd64 md: running: <sdc2><sdb2><sda2>
Apr 10 22:30:10 amd64 md: kicking non-fresh sdc2 from array!
Apr 10 22:30:10 amd64 md: unbind<sdc2>
Apr 10 22:30:10 amd64 md: export_rdev(sdc2)
Apr 10 22:30:10 amd64 raid5: device sdb2 operational as raid disk 0
Apr 10 22:30:10 amd64 raid5: device sda2 operational as raid disk 2
Apr 10 22:30:10 amd64 raid5: allocated 3202kB for md0
Apr 10 22:30:10 amd64 raid5: raid level 5 set md0 active with 2 out of 3 devices, algorithm 0
Apr 10 22:30:10 amd64 RAID5 conf printout:
Apr 10 22:30:10 amd64 --- rd:3 wd:2 fd:1
Apr 10 22:30:10 amd64 disk 0, o:1, dev:sdb2
Apr 10 22:30:10 amd64 disk 2, o:1, dev:sda2
Apr 10 22:30:10 amd64 md: ... autorun DONE.
Apr 10 22:30:10 amd64 md: Unknown device name: hde3
Apr 10 22:30:10 amd64 RAMDISK: Couldn't find valid RAM disk image starting at 0.
Apr 10 22:30:10 amd64 kjournald starting.  Commit interval 5 seconds
Apr 10 22:30:10 amd64 EXT3-fs: mounted filesystem with ordered data mode.
Apr 10 22:30:10 amd64 VFS: Mounted root (ext3 filesystem) readonly.
Apr 10 22:30:10 amd64 Mounted devfs on /dev
Apr 10 22:30:10 amd64 Freeing unused kernel memory: 264k freed
Apr 10 22:30:10 amd64 Adding 3148536k swap on /dev/md0.  Priority:-1 extents:1
Apr 10 22:30:10 amd64 EXT3 FS on md1, internal journal
Apr 10 22:30:10 amd64 kjournald starting.  Commit interval 5 seconds
Apr 10 22:30:10 amd64 EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
Apr 10 22:30:10 amd64 EXT3 FS on md2, internal journal
Apr 10 22:30:10 amd64 EXT3-fs: mounted filesystem with ordered data mode.
Apr 10 22:30:10 amd64 kjournald starting.  Commit interval 5 seconds
Apr 10 22:30:10 amd64 EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
Apr 10 22:30:10 amd64 EXT3 FS on md3, internal journal
Apr 10 22:30:10 amd64 EXT3-fs: mounted filesystem with ordered data mode.


any further help appreciated.
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Sun Apr 11, 2004 1:47 am    Post subject: Reply with quote

Ok let's see
md0 using partitions sd?2 (bca) is using algorithm 0, whereas the others use algorithm 2, this is presumably the consequence of missing put the "parity-algorithm left-symmetric" in the raidtab.
But md0 is just swap so though it may be a useful test of any proposed fix, it is not too important, as already mentioned there are better ways to organise swap and we can just delete it and start again with no loss.

md1 using partitions sd?3 (bca) is your root which is the one we really need to fix
The thing to note is
Code:

Apr 10 22:30:10 amd64 md: running: <sdc3><sdb3><sda3>
Apr 10 22:30:10 amd64 md: kicking non-fresh sda3 from array!
Apr 10 22:30:10 amd64 md: unbind<sda3>
Apr 10 22:30:10 amd64 md: export_rdev(sda3)

which occurs with the swap partition also.

Other thing to note is that when starting to assemble the md0 array it seems to have decided one disk has failed:
Code:

Apr 10 22:30:10 amd64 --- rd:3 wd:2 fd:1
Apr 10 22:30:10 amd64 disk 0, o:1, dev:sdb3
Apr 10 22:30:10 amd64 disk 1, o:1, dev:sdc3

ie disk sda is not even considered

All this is, I believe, part of the autodetect process from within the kernel
The init script then checks the raidtab and has a problem with a reference to hde3
Code:

Apr 10 22:30:10 amd64 RAID5 conf printout:
Apr 10 22:30:10 amd64 --- rd:3 wd:2 fd:1
Apr 10 22:30:10 amd64 disk 0, o:1, dev:sdb2
Apr 10 22:30:10 amd64 disk 2, o:1, dev:sda2
Apr 10 22:30:10 amd64 md: ... autorun DONE.
Apr 10 22:30:10 amd64 md: Unknown device name: hde3

It could be that there really is something wrong with sda3 in which case you might try
Code:

badblocks -f /dev/sda3

But I think the problem may have started with the confusion over the disk drive naming, so that something in the persistent-superblock is not identifying the partitions correctly.

Now googling around for " md: kicking non-fresh from array!" it seems the solution is probably
Get mdadm and use
Code:

mdadm --assemble --force /dev/md0 /dev/sdb2 /dev/sdc2 /dev/sda2   

mdadm --assemble --force /dev/md1 /dev/sdb3 /dev/sdc3 /dev/sda3

I am not sure is this will fix the swap array because of the separate parity-algorithm problem but obviously it would be good to test it there before trying with the root partition
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Sun Apr 11, 2004 12:18 pm    Post subject: Reply with quote

i allready had mdadm. as it doesn't seem to work with a running array i booted with the livecd and tried your suggestions. unfortunately the livecd has no mdadm tools. so i chrooted to my system and tried it from there with no avail. i remade my swap array with
Code:
mkraid -R /dev/md0
and it's now working with all 3 drives up. what i should do with my root array i still don't know. it's strange that each of my arrays has one drive down within the livecd environment (with the exception of a newly created array - alltough this happened from the beginning this is still a strange behaviour). i just run
Code:
badblocks -f /dev/sda3
which hasn't returned anything. so i guess there's nothing wrong with the drive.
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Sun Apr 11, 2004 12:44 pm    Post subject: Reply with quote

Well I'm glad you have something fixed.

It seems like the confusion between hde and hda is the real problem. If you create within the live cd its ok, but seems to fail when switching to your installed system if I've understood you rightly.

There is some useful information about mdadm at
http://www.linuxdevcenter.com/pub/a/linux/2002/12/05/RAID.html

Quote:
Managing Arrays

Using Manage mode you can add and remove disks to a running array. This is useful for removing failed disks, adding spare disks, or adding replacement disks. Manage mode can also be used to mark a member disk as failed. Manage mode replicates the functions of raidtools programs such as raidsetfaulty, raidhotremove, and raidhotadd.

For example, to add a disk to an active array, replicating the raidhotadd command:

Code:
# mdadm /dev/md0 --add /dev/sdc1


Or, to remove /dev/sdc1 from /dev/md0 try:

Code:
# mdadm /dev/md0 --f ail /dev/sdc1 --remove /dev/sdc1


Notice that I first mark /dev/sdc1 as failed and then remove it. This is the same as using the raidsetfaulty and raidhotremove commands with raidtools. It's fine to combine add, fail, and remove options on a single command line as long as they make sense in terms of array management. So you have to fail a disk before removing it, for example.


Perhaps you could try this on the md1 array, I think you will need to boot your installation rather than use the live cd or chroot for this. As I've said I think it all stems from the difference between live cd and your installed system about the disk naming, at some point a raid tab with the wrong information has muddled things up.

If using mdadm dosn't work. perhaps the only solution is to create a new partition withouut raid. copy / to that, modify your fstab and bootloader to boot from the new /, then from within that system, recreate your raid array and copy / back to it. Well there is always reinstalling :-(
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Sun Apr 11, 2004 2:02 pm    Post subject: Reply with quote

yeah, you're the man. you saved me a lot of time (i was just about to copy my data to another disk and to rebuild the array when i noticed that in the mean time one drive of my md2 (/home) array was down too).
so, a brief overview of what i did:
Code:
 mdadm /dev/md2 --add /dev/sda5
Code:
 mdadm /dev/md1 --add /dev/sda3
because sda5 and sda3 were the drives that have been down. it's currently syncing my disk but i'm pretty convinced that this will solve my problem.

i found out that the problem which resulted in an output
Code:
 Apr 10 22:30:10 amd64 md: Unknown device name: hde3
was a wrong grub.conf. i didn't mention but i accidently lost my /boot partition with my grub.conf and had no up-to-date backup. so it looked like this:
Code:
 title=Gentoo kernel-2.6.3
  root (hd0,0)
  kernel (hd0,0)/2.6.3-gentoo-r2/bzImage root=/dev/md1 md=1,/dev/hde3,/dev/hdg3,/dev/sda3
i don't know why this worked (i couldn't boot my system a long time because grub recognizes my hd's as sdX unlike to the livecd environment) - it shouldn't boot the system - but i corrected the drives to
Code:
kernel (hd0,0)/2.6.3-gentoo-r2/bzImage root=/dev/md1 md=1,/dev/sda3,/dev/sdb3,/dev/sdc3
anyway, and the previously mentioned error disappeared.

thanks a lot for investigating yourself to help me solve this issue.
Back to top
View user's profile Send private message
slais-sysweb
Apprentice
Apprentice


Joined: 14 Jun 2002
Posts: 221
Location: London

PostPosted: Sun Apr 11, 2004 4:45 pm    Post subject: Reply with quote

Glad to here it looking better!

I'm a bit puzzled by your grub.conf line.
Code:

  kernel (hd0,0)/2.6.3-gentoo-r2/bzImage root=/dev/md1 md=1,/dev/hde3,/dev/hdg3,/dev/sda3

and its correction
Code:

kernel (hd0,0)/2.6.3-gentoo-r2/bzImage root=/dev/md1 md=1,/dev/sda3,/dev/sdb3,/dev/sdc3


Grub uses its own designation of drives (hd0,0) which are independent of linux. (the best way to set up grub is to use the tab completion feature to see what drives you have according to grub's scheme of naming)
The lines in your grub.conf refering to md./dev/... are passed to the kernel and not used by grub.
In particular I've never needed a line like
Code:
md=1,/dev/sda3,/dev/sdb3,/dev/sdc3

so what is it supposed to do?

One other suggestion: I've never seen the boint of booting from raid in Gentoo as the boot partition is only mounted at boot. On the other hand I do think it useful to partition all the disks the same. So I create a /boot on each disk and manually mount and make copies of /boot to the spare partition(s). This provides an backup if the boot partition fails: either edit the boot details on the grub command when it boots, or if the mbr fails, boot from cd/floppy and setup grub for the backup disk.
_________________
--
djc
sysweb SLAIS UCL
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Sun Apr 11, 2004 6:00 pm    Post subject: Reply with quote

Quote:
so what is it supposed to do?

i don't know anymore. i've read that somewhere around and finally was able to boot my system (well, booting is the wrong word, because i could boot but not mount the root partition). https://forums.gentoo.org/viewtopic.php?t=145738&highlight=
i just booted without this line
Code:
md=1,/dev/sda3,/dev/sdb3,/dev/sdc3
hence it is not necessary. my mistake was (referring to the above mentioned thread) that i had a wrong fstab/raidtab. good to know!

Quote:
One other suggestion: I've never seen the boint of booting from raid in Gentoo as the boot partition is only mounted at boot. On the other hand I do think it useful to partition all the disks the same. So I create a /boot on each disk and manually mount and make copies of /boot to the spare partition(s). This provides an backup if the boot partition fails: either edit the boot details on the grub command when it boots, or if the mbr fails, boot from cd/floppy and setup grub for the backup disk.

this is exactly the way i did it. i didn't make that clear, but thanks anyway for your hint.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum