View previous topic :: View next topic |
Author |
Message |
seba_gentoo n00b
Joined: 21 Jul 2016 Posts: 7 Location: Bavaria
|
Posted: Thu Jul 21, 2016 9:59 am Post subject: re-creating a raid 1 on new system |
|
|
Dear forum members,
I was forced to set up a new gentoo system. I had an old one with software (mdadm) raid 1 running.
I have issues with creating and mounting this two disks to a raid and make it mountable.
I, of course, don't want to lose my data stored on this drives.
#cat /proc/mdstat
Personalities : [raid1]
md127 : inactive sdb1[1](S)
976630488 blocks super 1.2
md0 : active raid1 sda1[0]
976761472 blocks [2/1] [U_]
bitmap: 1/8 pages [4KB], 65536KB chunk
unused devices: <none>
#mount /dev/md0 /mnt/md0
mount: frong fs type, bad option, bad superblock on /dev/md0
missing codepage or helper program, or other error
#dmesg| tail
[ 3421.400766] EXT3-fs error (device md0): ext3_check_descriptors: Block bitmap for group 1 not in group (block 0)!
[ 3421.454371] EXT3-fs (md0): error: group descriptors corrupted
Could someone help me out of this problem?
Thanks a lot!
sebastian |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Thu Jul 21, 2016 10:21 am Post subject: |
|
|
file -s /dev/md0?
mdadm --examine /dev/sd*? |
|
Back to top |
|
|
pilla Bodhisattva
Joined: 07 Aug 2002 Posts: 7729 Location: Underworld
|
Posted: Thu Jul 21, 2016 10:58 am Post subject: |
|
|
Hello,
Looks like you got some nasty file system corruption in your RAID.
First, sdb1 seems to be a dead disk. So I would try first to fsck it directly, then try to mount it without using the RAID system.
Code: |
fsck.ext3 -f -p -v /dev/sdb1
mkdir /mnt/sdb1
mount /dev/sdb1 /mnt/sdb1
|
There are other flags for fsck, but I think those are a good start (force even if clean, fix automatically, verbose).
If it does not work, you can try it with sda1. Software RAID should not have special stuff in the disk for RAID 1, so you should be able to mount the disks without it separately. _________________ "I'm just very selective about the reality I choose to accept." -- Calvin |
|
Back to top |
|
|
seba_gentoo n00b
Joined: 21 Jul 2016 Posts: 7 Location: Bavaria
|
Posted: Thu Jul 21, 2016 11:44 am Post subject: |
|
|
file -s:
/dev/md0: Linux rev 1.0 ext3 filesystem data, UUID=4fefcfa7-16d4-48fc-84d2-44d5adbc3694 (errors) (large files)
examine sda1:
/dev/sda1:
Magic : a92b4efc
Version : 0.90.00
UUID : 3836eeda:095a9bde:94d4a16f:27c4d8e2 (local to host seba_desktop)
Creation Time : Wed Jul 20 23:27:31 2016
Raid Level : raid1
Used Dev Size : 976761472 (931.51 GiB 1000.20 GB)
Array Size : 976761472 (931.51 GiB 1000.20 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 0
Update Time : Thu Jul 21 13:52:13 2016
State : clean
Internal Bitmap : present
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
Checksum : 90af2c78 - correct
Events : 27
Number Major Minor RaidDevice State
this 0 8 1 0 active sync /dev/sda1
0 0 8 1 0 active sync /dev/sda1
1 1 0 0 1 faulty removed
examine sdb1:
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : ba924d1c:a85f2460:dc03db17:da419e2e
Name : seba_desktop:127 (local to host seba_desktop)
Creation Time : Wed Jul 13 10:58:39 2016
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 1953260976 (931.39 GiB 1000.07 GB)
Array Size : 976630464 (931.39 GiB 1000.07 GB)
Used Dev Size : 1953260928 (931.39 GiB 1000.07 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=48 sectors
State : clean
Device UUID : 5d939af1:fd090aec:6f6218c6:1c334139
Internal Bitmap : 8 sectors from superblock
Update Time : Wed Jul 20 23:23:12 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 5d3aed23 - correct
Events : 2398
Device Role : Active device 1
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
regards,
sebastian |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Thu Jul 21, 2016 12:10 pm Post subject: |
|
|
These don't really make much sense. One is 0.90 metadata, the other is 1.2 metadata. Both state they were created just a day/a week ago. So neither of them is "an old one". Did you mess around with mdadm --create?
sdb1 has a data offset of 262144 sectors, you can try and see if there is anything that makes sense at this offset.
Code: |
losetup --find --show --read-only --offset=$((262144*512)) /dev/sdb1
file -s /dev/loop?
mount -o ro /dev/loop? /mnt/somewhere
|
|
|
Back to top |
|
|
seba_gentoo n00b
Joined: 21 Jul 2016 Posts: 7 Location: Bavaria
|
Posted: Thu Jul 21, 2016 6:39 pm Post subject: |
|
|
[quote="frostschutz"]These don't really make much sense. One is 0.90 metadata, the other is 1.2 metadata. Both state they were created just a day/a week ago. So neither of them is "an old one". Did you mess around with mdadm --create?
sdb1 has a data offset of 262144 sectors, you can try and see if there is anything that makes sense at this offset.
[code]
losetup --find --show --read-only --offset=$((262144*512)) /dev/sdb1
/dev/loop0
file -s /dev/loop?
/dev/loop0: data
mount -o ro /dev/loop? /mnt/somewhere
mount: unknown filesystem type 'linux_raid_member'
[/code][/quote]
Yes, I mess around with mdadm.
First I tried to create a new raid according to this manual:
https://wiki.gentoo.org/wiki/Complete_Handbook/Software_RAID
I didn't make the step with mkfs because I don't want to lose my data.
Than I found a post and tried to create a new raid with one missing disk.
How to (which mode) remove all the settings and start from scratch?
Thanks for your efforts
sebastian |
|
Back to top |
|
|
seba_gentoo n00b
Joined: 21 Jul 2016 Posts: 7 Location: Bavaria
|
Posted: Thu Jul 21, 2016 6:57 pm Post subject: |
|
|
[quote="pilla"]Hello,
Looks like you got some nasty file system corruption in your RAID.
First, sdb1 seems to be a dead disk. So I would try first to fsck it directly, then try to mount it without using the RAID system.
[code]
fsck.ext3 -f -p -v /dev/sdb1
[b]/dev/sdb1: Note: if several inode or block bitmap blocks or part
of the inode table require relocation, you may wish to try
running e2fsck with the '-b 32768' option first. The problem
may lie only with the primary block group descriptors, and
the backup block group descriptors may be OK.
/dev/sdb1: Block bitmap for group 1 is not in group. (block 0)[/b]
mkdir /mnt/sdb1
mount /dev/sdb1 /mnt/sdb1
[/code]
There are other flags for fsck, but I think those are a good start (force even if clean, fix automatically, verbose).
If it does not work, you can try it with sda1. Software RAID should not have special stuff in the disk for RAID 1, so you should be able to mount the disks without it separately.[/quote]
Sorry, but mount is not possible. |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Thu Jul 21, 2016 7:31 pm Post subject: |
|
|
seba_gentoo wrote: |
Yes, I mess around with mdadm.
I didn't make the step with mkfs because I don't want to lose my data.
|
Unfortunately, mdadm --create already loses you data. You don't ever use mdadm --create if there is data you want to keep on it (unless you know really really really well what you're doing and even then you usually need a lot more options than a regular --create to make things work). If the RAID is already there all you need is --assemble, if that does not work you --examine and ask for help.
mdadm writes metadata to the start of the disk (1.1, 1.2 metadata) or to the end of the disk (0.90, 1.0 metadata) and either will overwrite your filesystem data in those places - depending on where that filesystem was before.
Unfortunately I'm not sure if/how this can be repaired in your particular case. fsck is not a tool that magically fixes things... fsck makes a lot of guesswork and then writes to the disk, if the guesses were wrong, fsck caused even more damage instead of fixing anything. That's why I sometimes refer to fsck as being a file system "crunch killer" instead of a "checker".
Quote: | How to (which mode) remove all the settings and start from scratch? |
mdadm --zero-superblock would remove mdadm metadata, then you could try with whatever tools (testdisk/photorec, fsck with and without backup-superblocks, ...).
Before you experiment with fsck, consider putting an overlay/snapshot on it (or pulling a full copy) so you can undo any further damages if your experiments go wrong.
https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID#Making_the_harddisks_read-only_using_an_overlay_file (and overlay functions below that) |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54030 Location: 56N 3W
|
Posted: Thu Jul 21, 2016 8:01 pm Post subject: |
|
|
seba_gentoo,
Tell us exactly what you did. Posting /root/.bash_history may be useful.
What sort of raid superblock did you have originally?
If you originally used version 0.9, that has the raid data at the end of the drive and the filesystem starting in the usual place, when you overwrite that with a version 1.2 raid superblock, you destroy the start of the filesystem.
In particular, the primary filesystem superblock is gone. However, it may still mount, if you tell mount to use an alternate superblock.
Whenever something bad happens to a HDD, the last thing to do is to run anything that writes to the disk. It can only make a bad situation worse.
Data recovery requires a bit of guesswork, so the first step is to make an image, so you have an undo facility in case you make a wrong guess.
Most of your data will still be there but you will not be able to recover any of it 'in place'.
If we get something to mount and if files are still there, you will need to copy them. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
seba_gentoo n00b
Joined: 21 Jul 2016 Posts: 7 Location: Bavaria
|
Posted: Thu Jul 21, 2016 8:35 pm Post subject: |
|
|
[quote="NeddySeagoon"]seba_gentoo,
Tell us exactly what you did. Posting /root/.bash_history may be useful.
What sort of raid superblock did you have originally?
I did it really bad!: (some most important things from bash_history)
mdadm --create /dev/md0 --auto md --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
mount /dev/md0 /mnt/stick/
mdadm --assemble --run --level=1 /dev/md0 --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm --assemble --run /dev/md0 --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm --assemble --run /dev/md0
mdadm --assemble --run /dev/md0 /dev/sda1
mdadm --assemble --run /dev/md0 /dev/sdb1
mdadm --stop /dev/md127
mdadm --stop /dev/md0
mdadm --create /dev/md127 --auto md --level=1 --raid-devices=2 /dev/sda1
mdadm -A /dev/md127 -f --update-summaries /dev/sda1 /dev/sdb1
mdadm --examine /dev/sda1 /dev/sdb1
mdadm -S /dev/md127
mdadm -S /dev/md0
fdisk -l /dev/sda
mdadm -E /dev/sda1
mdadm -E /dev/sdb1
mdadm -A /dev/md0 /dev/sdb1 missing
mdadm --create /dev/md0 --metadata=0.9 --raid-devices=2 --level=1 /dev/sdb1
fsck.ext2 /dev/sda1
fsck.ext2 /dev/sdb1
e2fsck -b 32768 /dev/md127
e2fsck -b 32768 /dev/sda1
e2fsck -b 32768 /dev/sdb1
I'll try to secure some data with ddrescue before I do something with mdadm.
Thanks guys! |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54030 Location: 56N 3W
|
Posted: Thu Jul 21, 2016 9:46 pm Post subject: |
|
|
seba_gentoo,
Code: | mdadm --create /dev/md0 --auto md --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1 |
This command would write new raid version 1.2 metadata to the end of both drives, destroying what was already there.
If it was already version 1.2 metadata no user data is destroyed. It also syncs the two parts of the mirror, making them identical.
I don't know how that is done.
You did not make any filesystem - thats good.
The you did Code: | mount /dev/md0 /mnt/stick/ |
Did that command work?
If your original raid used version 1.2 metadata, it might. The filesystem may still have been there.
If you originally had 0.9 metadata, the --create destroyed the start of the filesystem and the mount will fail.
What to try ...
If your raid superblocks were originally 0.9 then
Code: | mount -o ro,sb=131072 /dev/sda1 /mnt/stick | will try to use the first backup fs superblock to mount the filesystem.
There are other backup superblocks too
Now, if you originally had 1.2 raid superblocks, the first backup superblock is 262144 blocks further down the drive.
Looking at those numbers 262144 is bigger than 131072 so the first backup fs superblock may be gone as well. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
seba_gentoo n00b
Joined: 21 Jul 2016 Posts: 7 Location: Bavaria
|
Posted: Fri Jul 22, 2016 7:17 am Post subject: |
|
|
NeddySeagoon wrote: | seba_gentoo,
The you did Code: | mount /dev/md0 /mnt/stick/ |
Did that command work?
No, the mount command never worked. There was always a problem with wrong fs type, bad option etc.
If your raid superblocks were originally 0.9 then
Code: | mount -o ro,sb=131072 /dev/sda1 /mnt/stick | will try to use the first backup fs superblock to mount the filesystem.
There are other backup superblocks too
In this case mount replies with:
Code: | mount: unknown filesystem type 'linux_raid_member'
|
Now, if you originally had 1.2 raid superblocks, the first backup superblock is 262144 blocks further down the drive.
Looking at those numbers 262144 is bigger than 131072 so the first backup fs superblock may be gone as well. |
Thank you for your proposals.
sebastian
I am about to order a new drive and secure the image if this disk and to go to more risky things. |
|
Back to top |
|
|
seba_gentoo n00b
Joined: 21 Jul 2016 Posts: 7 Location: Bavaria
|
Posted: Sat Aug 06, 2016 8:44 pm Post subject: Thanks |
|
|
Hello Forum Members.
I succeeded. I used testdisk to recover the data.
I created new raid and made new file system.
Thanks for all your proposals.
stay tuned,
sebastian |
|
Back to top |
|
|
pilla Bodhisattva
Joined: 07 Aug 2002 Posts: 7729 Location: Underworld
|
Posted: Sun Aug 07, 2016 11:27 am Post subject: |
|
|
Good news!
Just remember that RAID is not a substitute for backup. You should backup your important data and store somewhere else. You could even use one of the many cloud services with encryption. _________________ "I'm just very selective about the reality I choose to accept." -- Calvin |
|
Back to top |
|
|
|