Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
RAID troubleshooting
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
Belliash
Advocate
Advocate


Joined: 24 Nov 2004
Posts: 2503
Location: Wroclaw, Poland

PostPosted: Fri Aug 17, 2012 2:52 pm    Post subject: RAID troubleshooting Reply with quote

Hello guys,

Today i noticed a problem with my RAID arrays, while booting it says:
Code:
mdadm start with 1 drive out of 2
mdadm start with 1 drive out of 2


and from dmesg:
Code:
[    0.000000] Command line: root=/dev/md2 rootfstype=ext4 hpet64 softraid raid=noautodetect initrd=/initramfs.cpio.gz BOOT_IMAGE=/vmlinuz-3.5.2-regen2
[    0.000000] Kernel command line: root=/dev/md2 rootfstype=ext4 hpet64 softraid raid=noautodetect initrd=/initramfs.cpio.gz BOOT_IMAGE=/vmlinuz-3.5.2-regen2
[    0.471970] md: raid0 personality registered for level 0
[    0.472030] md: raid1 personality registered for level 1
[    2.037241] md: md1 stopped.
[    2.049169] md: bind<sda2>
[    2.049515] md: bind<sda1>
[    2.049556] md: kicking non-fresh sda2 from array!
[    2.049566] md: unbind<sda2>
[    2.100057] md: export_rdev(sda2)
[    2.100946] md/raid1:md1: active with 1 out of 2 mirrors
[    2.100994] md1: detected capacity change from 0 to 131559424
[    2.102774]  md1: unknown partition table
[    2.550768] md: md2 stopped.
[    2.552514] md: bind<sdb2>
[    2.553409] md/raid1:md2: active with 1 out of 2 mirrors
[    2.553450] md2: detected capacity change from 0 to 497973713920
[    2.556494] md: Autodetecting RAID arrays.
[    2.556513] md: could not open unknown-block(8,1).
[    2.556835] md: invalid raid superblock magic on sda2
[    2.556842] md: sda2 does not have a valid v0.90 superblock, not importing!
[    2.558910] md: invalid raid superblock magic on sdb1
[    2.558916] md: sdb1 does not have a valid v0.90 superblock, not importing!
[    2.558933] md: could not open unknown-block(8,18).
[    2.558938] md: Scanned 4 and added 0 devices.
[    2.558941] md: autorun ...
[    2.558944] md: ... autorun DONE.
[    2.574741]  md2: unknown partition table
[    3.300119] EXT4-fs (md2): mounted filesystem with journalled data mode. Opts: (null)
[   10.524459] systemd-udevd[644]: starting version 188
[   16.616246] EXT4-fs (md2): re-mounted. Opts: acl,errors=remount-ro


I use better-initramfs to assemble the arrays... and it worked, but somehow does not work now (i have no idea how long it takes - notices this today).
The array has been created over a year ago.
/dev/md1 uses superblock v1.0
/dev/md2 uses superblock v1.2
and i use extlinux as a bootloader...

Code:
Personalities : [raid0] [raid1]
md2 : active raid1 sdb2[1]
      486302455 blocks super 1.2 [2/1] [_U]
     
md1 : active raid1 sda1[0]
      128476 blocks super 1.0 [2/1] [U_]
     
unused devices: <none>


I wanted to add missing devices back to array:
Code:
> mdadm /dev/md1 --add /dev/sdb1
mdadm: /dev/sdb1 reports being an active member for /dev/md1, but a --re-add fails.
mdadm: not performing --add as that would convert /dev/sdb1 in to a spare.
mdadm: To make this a spare, use "mdadm --zero-superblock /dev/sdb1" first.
> mdadm --zero-superblock /dev/sdb1
> mdadm /dev/md1 --add /dev/sdb1
mdadm: added /dev/sdb1


and then:

Code:
> cat /proc/mdstat
Personalities : [raid0] [raid1]
md2 : active raid1 sdb2[1]
      486302455 blocks super 1.2 [2/1] [_U]
     
md1 : active raid1 sdb1[2] sda1[0]
      128476 blocks super 1.0 [2/2] [UU]
     
unused devices: <none>



Looks like it works.. ok. But if i reboot it happens all again :(
Any ideas?

Thanks in advance!
_________________
Asio Software Technologies
Belliash IT Weblog
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Fri Aug 17, 2012 5:55 pm    Post subject: Reply with quote

When you shutdown/reboot do you see the array being disassembled and shutdown cleanly (meaning, no warnings about open files, etc.)?

Are you sure you're not running anything on the drives directly and not using /dev/md* ?

Come to think of it, I'm not sure what I actually do with my machine when trying to update grub stage1 on my raid1 bootdisk...hmmm...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Belliash
Advocate
Advocate


Joined: 24 Nov 2004
Posts: 2503
Location: Wroclaw, Poland

PostPosted: Fri Aug 17, 2012 10:04 pm    Post subject: Reply with quote

I said I'm using extlinux - grub sucks, does not support superblock newer than 0.90.

I have launched livecd and assembled both arrays... now i got this:

Code:
Personalities : [raid0] [raid1]
md2 : active raid1 sda2[2] sdb2[1]
      486302455 blocks super 1.2 [2/2] [UU]
     
md1 : active raid1 sda1[0]
      128476 blocks super 1.0 [2/1] [U_]
     
unused devices: <none>


Code:
[    0.000000] Command line: root=/dev/md2 rootfstype=ext4 hpet64 softraid raid=noautodetect initrd=/initramfs.cpio.gz BOOT_IMAGE=/vmlinuz-3.5.2-regen2
[    0.000000] Kernel command line: root=/dev/md2 rootfstype=ext4 hpet64 softraid raid=noautodetect initrd=/initramfs.cpio.gz BOOT_IMAGE=/vmlinuz-3.5.2-regen2
[    0.511928] md: raid0 personality registered for level 0
[    0.511988] md: raid1 personality registered for level 1
[    2.065046] md: md1 stopped.
[    2.067217] md: bind<sda1>
[    2.068158] md/raid1:md1: active with 1 out of 2 mirrors
[    2.068205] md1: detected capacity change from 0 to 131559424
[    2.069980]  md1: unknown partition table
[    2.507333] md: md2 stopped.
[    2.509366] md: bind<sdb2>
[    2.509698] md: bind<sda2>
[    2.510601] md/raid1:md2: active with 2 out of 2 mirrors
[    2.510643] md2: detected capacity change from 0 to 497973713920
[    2.513627] md: Autodetecting RAID arrays.
[    2.513645] md: could not open unknown-block(8,1).
[    2.513656] md: could not open unknown-block(8,2).
[    2.514034] md: invalid raid superblock magic on sdb1
[    2.514041] md: sdb1 does not have a valid v0.90 superblock, not importing!
[    2.514062] md: could not open unknown-block(8,18).
[    2.514067] md: Scanned 4 and added 0 devices.
[    2.514069] md: autorun ...
[    2.514072] md: ... autorun DONE.
[    2.892281]  md2: unknown partition table
[    3.960826] EXT4-fs (md2): mounted filesystem with journalled data mode. Opts: (null)
[   14.697151] systemd-udevd[643]: starting version 188
[   23.042659] EXT4-fs (md2): re-mounted. Opts: acl,errors=remount-ro



so this helped for md2, but md1 still cannot be assembled during bootup


I found something weird...


Code:
Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63      257039      128488+  fd  Linux raid autodetect
/dev/sda2          257040   972864269   486303615   fd  Linux raid autodetect
/dev/sda3       972864270   976773167     1954449   83  Linux


Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *          63      257039      128488+  fd  Linux raid autodetect
/dev/sdb2          257040   972864269   486303615   fd  Linux raid autodetect
/dev/sdb3       972864270   976773167     1954449   83  Linux


looks ok, but if i launch cfdisk...

Code:
sda1                            Boot                           Primary                    ext3                                       [BOOT]                                   131.61
sdb1                            Boot                           Primary                    ext3                                       [BOOT]                                   131.61



Why the hell ext3 and not linux_raid_m like sdb2:
Code:
sdb2                                                           Primary                    linux_raid_m                               [BellSrv:2]                           497974.91


what the hell? 8O
_________________
Asio Software Technologies
Belliash IT Weblog
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9645
Location: almost Mile High in the USA

PostPosted: Fri Aug 17, 2012 10:31 pm    Post subject: Reply with quote

You don't have to complain about the fact that I use grub on my machines, it's inconsequential - except the fact that grub as well as any bootloader needs to write to the disk to install the boot sector. When writing the boot sector it needs to write to both disks for a raid1 boot partition. If you write to one but not the other, this would cause problems. Not saying it's the problem you have but it's possible.

Ultimately it appears you're corrupting your raid volumes somehow. I've not seen a case with my RAIDs that are accessed through /dev/md* and shutdown before rebooting/powerdown to corrupt, so I'd look into that procedure first - not the issues of assembling. In the past I've had are corrupting via installing bootloaders or unclean RAID shutdown... I can't think of other possibilties at the moment that would cause later assembly issues.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Belliash
Advocate
Advocate


Joined: 24 Nov 2004
Posts: 2503
Location: Wroclaw, Poland

PostPosted: Fri Aug 17, 2012 11:18 pm    Post subject: Reply with quote

Bootloader is installed into MBR, and this region has nothing to do with software raid. It can be installed on single drive as well as on both.
This has nothing to do. Especially it was working over a year, and i have not reinstalled extlinux. It's not lilo, and does not require to write MBR again after any config file change.
_________________
Asio Software Technologies
Belliash IT Weblog
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum