| View previous topic :: View next topic |
| Author |
Message |
freedomlives
n00b
Joined: 19 Aug 2011
Posts: 32
Location: Slovakia
|
 Posted: Sat Jan 24, 2015 2:24 pm Post subject: recovering /etc enough to boot |
 |
|
Last night I suspended my system (running a tuxonice kernel). It was just to see if suspend was working after updates, as I´ve often had trouble getting suspend to work and didn´t really expect much. This morning I powered on, got a frozen X after it resumed from suspend, and had no other option but to pull the plug.
Upon restart I got a lot of errors of inode sizes (ext4) being wrong, etc., couldn´t find inittab, so couldn´t boot.
I now have systemrescuecd running, and I mounted the drives, all of my data (/home) was fine, but /etc couldn´t be accessed.
I unmounted and ran fsck.ext4 which found a lot of unconnected inodes (amongst other problems) that it moved to lost+found. The best I can tell, all that got corrupted on the drive was /etc which is now scatterred around lost+found in 1408 files and directories.
Can I just drop the /etc directory from a stage3 tarball in? then try to hunt through lost+found for important config files to restore?
Is it just wiser to backup /home and do a clean install? get rid of several years of built up cruft... |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Jan 24, 2015 2:35 pm Post subject: |
|
|
freedomlives,
Identifying bits of /etc in lost and found is relatively easy as they are almost all text files.
directories can be identified by the contents. File names may still be intact inside the directories.
I don't share your confidence that only /etc was trashed.
You will probably end up reinstalling anyway.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
freedomlives
n00b
Joined: 19 Aug 2011
Posts: 32
Location: Slovakia
|
| Posted: Sat Jan 24, 2015 2:52 pm Post subject: |
|
|
you´re probably right. I guess from a time perspective, if I re-install it will take awhile to compile some packages, but I can go do something else instead of guessing at names of /etc files and then have it boot only to hiccup at some other corrupted file.
As everyone else would say ¨I wish I had a backup¨, but I have the disks in RAID1, so I felt pretty safe! oops...
So, can I just delete all the directories except /home and jump to the ¨installing stage3¨ step of the installation?
(obviously backing up externally /home just to be safe...) |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Jan 24, 2015 3:04 pm Post subject: |
|
|
freedomlives,
As your filesystem metadata may be damaged, thats a maybe.
If /home is on its own partition, remake (mkfs) all of your other filesystems.
If home is on / you will need to make a backup first.
Personally, I would go for a clean start.
There is a useful shortcut for you. Save /var/lib/portage/world and move it to your new install.
Once you can reboot, you can do to get back all of the apps in your world file without trying to remember what you had installed.
The lesson from all this is that raid is not backup.
You say you have raid?
What raid level? ... there is another shortcut if its not raid0.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
freedomlives
n00b
Joined: 19 Aug 2011
Posts: 32
Location: Slovakia
|
| Posted: Sat Jan 24, 2015 3:10 pm Post subject: |
|
|
thanks for the advice so far
RAID1
It isn´t that I didn´t know that it is not a backup... but just any time in the past that I have had data loss, its been because of mechanical failure of a drive. and in that RAID1 has saved me once already.
I guess to stay a step ahead, I need not only to set up a NAS for backups, but drill a hole for ethernet and power into our safe and stick the thing the NAS in there.  |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Jan 24, 2015 4:00 pm Post subject: |
|
|
freedomlives,
I hope you are using mdadm raid, not fakeraid. This may apply to fakeraid but I don't know the details.
The idea is to split your raid1 into two sets of degraded raid1 sets, do the new install on the new degraded raid set then destroy the old raid set by using it to bring the new ones up to strenght.
Backup anything of value first.
You will have several /dev/mdX block devices composed of partitions from /dev/sda and /dev/sdb. /proc/mdstat will tell you more. Mine is below
| Code: | Personalities : [raid1] [raid6] [raid5] [raid4]
md127 : active raid5 sda6[0] sdd6[3] sdc6[2] sdb6[1]
2912833152 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
md126 : active raid5 sda5[0] sdd5[3] sdc5[2] sdb5[1]
15759360 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
md125 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
40064 blocks [4/4] [UUUU]
unused devices: <none> |
Using mdadm, fail and remove all of the /dev/sdb partitions from their raid sets. Now you have some degraded raid sets just using sda. That's what raid is all about.
It still works.
The partitions on /dev/sdb are no longer in use.
You can repartition sdb if you want to move space around.
use mdadm to make some new degraded raid1 sets with a partition from sdb and a missing partition. This missing partition will be replaced with a partition from sda later.
A few points to ponder.
If you use grub legacy, /boot must be an a raid metadata version 0.9 raid set.
If you use kernel raid auto detect you must use raid metadata version 0.9 everywhere.
With an initrd and grub2, the default raid metadata version 1.2 can be made to work
Install in the normal way using your new /dev/md* devices, so everything actually goes only onto sdb.
Test that you can reboot, /home should be empty, check /proc/mdstat. It should tell that you have two sets of degraded raids.
Copy over /home from the old raid, or restore it from your backup. Don't forget to preserve permissions.
/home will look a bit odd as ls will show user and group IDs, as the users/groups don't yet exist.
Copy over world.
Copy over lost+found if you want to poke about later.
Copy over anything else that might be useful.
Stop all the old raid sets, fail and remove the sda members.
Repartition sda to match sdb. This destroys the data on sda.
Use mdadm to add the partitions on sda to their correct /dev/md... nodes.
The raids will sync. (sda will be made a mirror of sdb). Watch progress in /proc/mdstat
Install the boot loader to sda - thats the bit outside the filesystem.
You can use the raid while it syncs.
Fix the rest of the damage.
You get to practice replacing a disk in a raid set at no extra cost.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
freedomlives
n00b
Joined: 19 Aug 2011
Posts: 32
Location: Slovakia
|
| Posted: Sat Jan 24, 2015 5:28 pm Post subject: |
|
|
I understand the advantage of file transfer-- it took about an hour to get the 300GB of files in /home onto an external USB drive, whereas I suppose a transfer between SATA drives should be faster.
But is there any other advantage to doing it this way?
Because otherwise I can just
| Code: | | mkfs.ext4 /dev/md126 |
chroot into there and install gentoo, then copy /home from the USB drive.
I know I´ll miss out on the RAID practice, but I had that practice two years ago when a drive failed, and I suspect by the next failure I´ll have forgotten anyway. 
thanks,
Andrew[/code] |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Jan 24, 2015 5:43 pm Post subject: |
|
|
freedomlives,
There is the advantage that you will actually have a backup of /home at all times
While your only copy is on the USB drive, you no longer have a backup.
Don't forget to copy lost+found, don't use that name for the destination and your world file.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
freedomlives
n00b
Joined: 19 Aug 2011
Posts: 32
Location: Slovakia
|
| Posted: Wed Jan 28, 2015 9:46 pm Post subject: |
|
|
Thank you NeddySeagoon for the help!
I got everything back up and running. It was especially helpful the advice about copying the world file.
I didn't do the way you suggested with RAID for restoring home though-- I had a second harddrive to connect to the USB adapter, so it was safe copying and restoring that way. 
The system actually seems to run and portage compiles a lot better now, as this cleaned out 7 years of configs, wherein there were probably lingering some less-than-perfect configurations I made in the past.
The one other thing I sort of wish I'd backed up was my kernel .config, but it wasn't the end of the world to spend an hour doing that over from scratch.
I have the lost+found backed up, and am going to hunt through it for any images or documents that might have gotten lost, because when I tarred it it came out at 1.5GB, which isn't just /etc
Anyway, thanks again for your quick replies and help! |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
