ddrescue & btrfs [done]

[idella4]

I have a post in kernel & hardware which has clearly gone cold.
I am trying to restore some data from a corrupted partition. It all started as a curiosity experiment to use btrfs.
Following another post by an advocate who had pursued btrfs closely, it appeared btrfs is somewhat of a lemon at this point. Its performance is riddled with errors and predictably my experimental partition broke.
The partition contained kernels and a distfiles folder of many past packages and a portage in an image file.
This was some months ago. The data is entirely non critical. As an exercise I'd just like to retrieve it and put it back and delete my corrupted btrfs file.
I used dd to copy the partition to a file gen2.img, 48.2 Gig. This is because the partition was 48.2 Gig at the time but it was only about half filled.
The btrfs utils' btrfsck is a lemon. It just doesn't work. After trying it I read more past posts and they confirm this.

Mounting gen2.img results in dealing with the failed state of the btrees and metadata.
Attempts to copy out data just produces errors and it hangs. Another post lead me to ddrescue.
ddrescue advises to copy out segments of the corrupted data of origin, such as the first 500 mb or 2 gig.
Doing this results in a a file ok but it's not usable. Mounting the output file, call it rescue.img, results in

It tries very hard to read the partition/drive you point it at and make an image. Naturally, this includes all the defects too.

[NeddySeagoon]

idella4,

You have a broken btrfs. When you use dd or ddrescue, you have a file on an ext4 filesystem containing a image of your broken btrfs.
There is no magic to convert a broken btrfs filestem to ext4.

Many filesystems store key metadats at the front of the filesystem. e.g. vfat. As long as you have everything up to the root directory, you can mount such a truncated filesystem. I think the same its true of ext2 and ext3. Its not a good idea to put the metadata at the front as it slows down access to the end of the filesystem, unless the metadata is cached. Maybe btrfs has distributed metadata.

In short, you fix your btrfs file system or go fishing for your data yourself, on the broken filesystem. They are really your only choices.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[idella4]

[NeddySeagoon]

idella4,

dd and ddrescue clone the area of a disk you point them to. The only difference is how they handle areas that cannot be read from the drive.
You have no such areas. Your btrfs filesystem underlying drive reads every block properly. There are no CRC errors or bad blocks.
Unfortunately, the filesystem metadata stored in some of these blocks is not correct, so mount won't work. No amount of low level copying of data will change that.

Emerge hexedit, make your image file read only, so you don't change it accidently and go fishing for

[idella4]

ok Neddy, thanks.

[NeddySeagoon]

idella4,

The kernel is hundreds of binary files - don't even think about it. True, they will all be in one directory but you have no idea if the directory structure is intact or not.
Nor do you know where to find it, nor how to read it when you do. Thats all documented of course but its not needed.

Once you have recovered the .config file, you can get the kernel sources, patch sets and use the recovered .config to recreate the kernel or am I missing something?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[idella4]

yes, sure, that's true. We are on the same page Neddy. That is exactly why I wasn't so hot on the hexedit option in the initial post.
It was fundamental that a kernel of that size and complexity would take weeks to copy out one file at a time, unless one could write a program in C to do it it one hit, but you point out the folder and the content can't be reliably tracked from faulty metadata.
I can see now you were thinking I particularly wanted the kernel configs again. They were all happily saved in their normal place in boot.
Like I said, I'm treating this as an exercise in going about data retrieval. At this point a number of basic points are in place re the btrfs and metadata.

ok, we both understand the whole kernels are re-emergeable. The distfiles are too, but only the one that exist in the current portage tree.

the gentoo package files, how to track and copy them out I think is all that's really left.
I presume just ascii search for .tar.bz2??
_________________
idella4@aus

[NeddySeagoon]

idella4,

The direct search and find fails for tarballs. There is nothing to search for and what there is has been compressed with bzip2.
The traditional way would be to search for the file name as this will normally find a directory entry, then decode the directory entry to find the start of the file and the length. Then copy that much data from that location on the filesystem. After the copy, test to see what you got.

You need to know how to read btrfs directory entries, what the reference point is for the start of the file ...
You don't even know that the dir entries are intact.

The last difficulty is that I'm fairly sure btrfs uses tail packing, so the last part of your file maybe somewhere else on the disk. With binary files, thats a disaster. With a kernel .config, you can always patch it up.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[idella4]

Hmmm, ok. [Your 2 IMs are still in my outbox]

[NeddySeagoon]

idella4,

By

[idella4]

Neddy,

good, ok. Just to tidy up, I am puzzled by

[NeddySeagoon]

idella4,

[idella4]

Neddy wrote

[idella4]

bump. This thread has just taken a turn.

ok Neddy, I am in business. I could have done this from the very beginning. It crossed my mind but I never got my self organised to actually do it.

[avendesora]

Hello,
Using dd to copy complete files is a bit strange. 'cp' is the more usual tool, or tar.
Have tried the usual tar pipe tar thing?

[idella4]

avendesora,

ok thanks. You posted while I was editing and adding the loop structure.

Re-read all the post.
This is a broken btrfs. How broken, very or only just can't tell. What I can tell is the following;

I have tried cp and tar. They don't work. As I said, it results in a hang.
After days of experimenting I've just discovered dd works as described.
Now it somehow gets out of jail. Using cp and tar, Control z fails. It's stuck and you cannot escape or unmount gen2.img.
Have to reboot to liberate the mounting of gen2.img

[avendesora]

All the variables are there.

Frankly, that ^Z does not make any sens. Plus, when are you killing those stopped dd's? Are they dying when you ^Z them, or when you bring them back from stopped state? Does it work if you ^C rather than ^Z? What do you have in dmesg when the process hangs?

BIG WARNING: the following, untested, will leave a stopped (^Z'ed) dd for each copied file.

[idella4]

avendesora,

that's it, very good.

[idella4]

avendesora,

almost there.

ok; I have made /mnt/fedora/store/gen2.img read only as Neddy suggested.
I have made a test folder at /mnt/gentoo/distfiles/ to give the destination location a clean empty folder and doesn't
have to compete with already present files.

[avendesora]

Sorry for the syntax. The correct if syntax would be:

[idella4]

avendesora,

very good. tx for the script, it does work
What doesn't work is the btrfs where it is broken. I've got up to

[idella4]

bump.
I suspect avendesora has left the building for now.
That's good. Finally completed the distfiles folder. The next step is to tackle a kernel.
This is not critical since a kernel can be re-emerged except that these kernels are mostly retired from the current portage tree.

To tackle a kernel I need one adjustment to the script. I need an equivalent script for directories.
In the uppermost directory of the kernel

export TARGET=//mnt/gentoo/linux-2.6.30-gentoo-r8
cd /mnt/ftp/linux-2.6.30-gentoo-r8
for [a folder] in ./*; do
if [ ! -folder $TARGET/$i ] ; then

I am only guessing intuitively. Trying

[NeddySeagoon]

idella4,

The script depends on the (broken) btrfs file system to find the blocks for dd to copy.
If file system errors ask dd to do something it can't do, it will stop with an error.
dd can take various action in the face of an error, none of which will recover the data, however it will keep going

[avendesora]

idella4: Real Bad Idea (tm).

Your ^Z'ed dd approach is only livable (barely) for a reasonable number of files since it creates an essentially dead process per file. The kernel source has thousands (>30k) of files - you might very well kill your system. Keep in mind that having a bunch of processes failing on I/O is generally something to avoid - you might hit weird bugs in other parts of the system leading to hard locks and such, which could be damaging to your other (valid) data.
Plus if any of the files necessary to build your kernel is damaged, all your efforts are wasted. (Added bonus: you might not see the damage, and the stuff might compile... booting that is mayhem - how good are your offline backups?)

Either:
- you managed to read the kernel tarballs, and they are not corrupt, in which case you're done (except for the config which you should try to recover separately if they are somewhere on that image).
Or:
- you don't have those, or they are corrupt, in which case you need to download them again (google will very likely find all the ones you're missing).

If you really want to try the wacky dd thing with a whole kernel tree, you'll have to do it yourself.
Hints: I think you'll need a two step approach. Find rebuild the directory structure in the target (possibly using 'find . -type d'), then 'copy' the files (possibly using 'find . -type f'). You could also try the more adventurous avenue of a recursive version of the previous script, which could build the target hierarchy as it goes.

Also don't use any kernel (compiled) image from that dead btrfs unless you have a way of knowing it's not damaged (like a hash of it stored somewhere other than on that failed partition). Probably won't boot at all if it's damaged, but you never know.


You keep the pieces if you hard-lock your live system, boot something that came straight out of that failed partition, or somehow manage to build a kernel from non-verified source.

[idella4]

Neddy

yes. agreed. I got to the end. It appears I only have a handful of broken files.