View previous topic :: View next topic |
Author |
Message |
gfunkmonk Tux's lil' helper
Joined: 10 Jul 2002 Posts: 129 Location: Freelandville, IN
|
Posted: Sun Sep 01, 2002 9:07 pm Post subject: Convert your filesystem [hopefully] without data loss.. |
|
|
This will work for these filesystems:
minix xfs jfs reiserfs ext2 ext3
Warning: This has caused at least 2 corrupt filesystmes (read posts below). Always backup your data. --pjp
OK, browse over to http://www.tux.org/pub/people/kent-robotti/looplinux/rip/
and download the bootable cd ISO, and burn it to a cd.
Reboot with the cd in the drive, and type:
Code: |
convertfs /dev/hdxx current_fs new_fs
|
After the conversion is done its probally a good idea to run fsck
Code: |
The 'reiserfsck' program is used to check and repair
a linux reiserfs filesystem.
The 'xfs_repair' program is used to repair a linux
xfs filesystem.
The 'fsck.jfs' program is used to check and repair
a linux jfs filesystem.
The 'e2fsck' program is used to check and repair
a linux ext2 or ext3 filesystem.
|
So something like:
Next, mount the filesystem,
Code: |
mount /dev/hdxx /mnt/linux
|
and chroot into the filesystem
Code: |
chroot /dev/hdxx /mnt/linux
|
and then proceede to edit /etc/fstab to use the correct filesystem, reboot and your done..
EDIT: Added warning, and modified title to include "hopefully." --pjp |
|
Back to top |
|
|
MoonWalker Guru
Joined: 04 Jul 2002 Posts: 510
|
Posted: Tue Oct 01, 2002 2:41 am Post subject: |
|
|
Just a warning... This totally corrupted my 1.4 system when I tried to change fs from xfs to reiserfs, so a total reinstall remained the only choice.
HW facts was ABIT BG7 Mobo, P4 1.8, 512MB RAM, IBM 120GB HD devided in 3 primary and 1 extended and 9 partitions all together.
So it didn't work for me, but it may do for you... your choise. _________________ /Joakim
Living on earth is expensive, but it includes a free trip around the sun
every year. |
|
Back to top |
|
|
nempo Guru
Joined: 16 Apr 2002 Posts: 360 Location: Linkoping, Sweden
|
Posted: Tue Oct 01, 2002 2:36 pm Post subject: |
|
|
Dosn't seem very reliable since it corrupted my ext3 root partition. |
|
Back to top |
|
|
delta407 Bodhisattva
Joined: 23 Apr 2002 Posts: 2876 Location: Chicago, IL
|
Posted: Tue Oct 01, 2002 5:11 pm Post subject: |
|
|
I think the moral of the story here is to have a backup before you attempt to do an in-place conversion between two very dissimilar filesystems. _________________ I don't believe in witty sigs. |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Tue Oct 01, 2002 5:25 pm Post subject: |
|
|
Edited gfunkmonk's post to include a warning. Sent a PM and mentioned he should feel free to change the warning, but that I thought one should be there. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
aardvark Guru
Joined: 30 Jun 2002 Posts: 576
|
Posted: Thu Oct 03, 2002 6:01 pm Post subject: |
|
|
delta407 wrote: | I think the moral of the story here is to have a backup before you attempt to do an in-place conversion between two very dissimilar filesystems. |
And when you're making a backup anyway, you might as well reformat the file system after backing up and placing it back on the new/different filesytem. (of course edit the fstab accordingly) You wont need to dl. that looplinux ISO.
This could be how:
1. boot from install CD. an press enter till you have a prompt.
2. mount the fs you want to backup: (choose '??' and "gentoobakfs" to your own needs)
Code: | mkdir /mnt/gentoobakfs
mount /dev/hd?? /mnt/gentoobakfs |
3. mount the place where the backup is to be written. (a partition with enough free space and a boot-cd-kernel-supported fs in the same manner as point 2. above)
4. cd to the mountpoint of point (2. )
5a. Backing up: Code: | tar -cvzpf /mnt/place_to_put/your_backup.tar.gz * |
(This will not pack any dot-files (hidden) in the 'root', but you normally don't have those.)
5b. Wait for it to complete (gzip is MUCH faster that bzip2 and only little larger)
6. unmount the partition you backed up: Code: | umount /mnt/gentoobakfs |
7. format the /dev/hd?? of point (2. ): Code: | mk(your_fs_of_choice) /dev/hd?? |
8. repeat point 2 and 4
9. Putting the backup back in place Code: | tar -xvzpf /mnt/place_to_put/your_backup.tar.gz |
10. If necessary, Code: | nano -w /mnt/gentoobakfs/etc/fstab |
11. Unmount everything and you should be set after rebooting without CD. |
|
Back to top |
|
|
guero61 l33t
Joined: 14 Oct 2002 Posts: 811 Location: Behind you
|
Posted: Sat Nov 02, 2002 1:35 am Post subject: |
|
|
Just thought I'd pass along that I just did the tarball conversion and it worked great, but some pointers I came up with before rebooting to the CD:
1. Make sure you have a kernel in place that will handle both filesystems
2. Don't forget to GRUB your /boot again; otherwise you'll get error 17.
Other than that, it was a flawless, lossless strategy; way to go! |
|
Back to top |
|
|
ejwahl Tux's lil' helper
Joined: 27 May 2002 Posts: 107 Location: Rhode Island
|
Posted: Sat Nov 02, 2002 6:13 pm Post subject: |
|
|
Here's code that worked for me when I backed up root:
Code: | find / -path /mnt -prune -o -path /dev -prune -o -path /proc -prune -o -print | cpio -pvdm /home/user/backup |
This copies everything in / except for /mnt, /dev and /proc to /home/user/backup.
HTH _________________ Erich J. Wahl
RLU 199500
ICQ 48100000 |
|
Back to top |
|
|
co-D n00b
Joined: 19 Mar 2003 Posts: 16
|
Posted: Mon Oct 27, 2003 2:05 am Post subject: |
|
|
This is what i made to make my backups easier, i'm a novice at scripting so revise it if you wish to use it, but i've backed up my system from it before when i've felt like reformatting root as a different filesystem.Just boot from the gentoo livecd (or any other bootable disk for that matter) and extract the tar.gz's to the new root and boot partitions.
You might have to reinstall your boot loader, but for some reason last time i did this it just worked.
Code: |
#!/bin/bash
archive='/mnt/xfs/backup'
directories='bin boot dev etc home lib mnt opt root sbin sys tmp usr var'
Date=`date +%F`
command='tar -cvzpf'
exclude='/usr/portage/distfiles/* /var/tmp/portage/* /mnt/*/*'
excludefile=$archive/$Date/excluded
if [ ! -d $archive/$Date ]
then
mkdir $archive/$Date
fi
if [ -e $archive/$Date/excluded ]
then
rm -f $archive/$Date/excluded
fi
for f in ${exclude}
do
echo ${f} >> $archive/$Date/excluded
done
echo -en '\E[40;37m'"\033[1m"Excluding Files:"
`cat $excludefile`\033[0m" # White
tput sgr0
echo ""
for d in ${directories}
do
if [ -e $archive/$Date/${d}.tar.gz ]
then
echo ""
echo -en '\E[40;36m'"\033[1m"Archive exists, skipping "$archive/$Date/${d}.tar.gz"..."\033[0m" # Cyan
tput sgr0
echo ""
else
echo ""
echo -en '\E[40;32m'"\033[1m"Creating "$archive/$Date/${d}.tar.gz"..."\033[0m" # Green
tput sgr0
echo ""
if [ $d == boot ]
then
mount /boot
$command $archive/$Date/${d}.tar.gz /${d} --exclude-from=$excludefile
umount /boot
else
$command $archive/$Date/${d}.tar.gz /${d} --exclude-from=$excludefile
fi
fi
done
echo -en '\E[40;31m'"\033[1m
"%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% NOTE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%"
"\ \ \ \ \ \ \ \ create /proc and /sys directories in new system \ \ "
"%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% NOTE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%"\033[0m" # Red
echo ""
tput sgr0
|
|
|
Back to top |
|
|
Trellph n00b
Joined: 26 Mar 2003 Posts: 19 Location: Houston
|
Posted: Wed Apr 21, 2004 3:37 am Post subject: |
|
|
I had me some fun with this. I thought like most applications that did this it would load itself into memory. I found out I was wrong when /bin was erased and it errored out
not to worry, /bin is easy to replace.
so I boot to a cd and do it, thinking that it loaded itself into memory but not mv rm cp and whatnot.
It made the image just fine, but then it stopped because it didn't load it's 2 other counterparts into memory. oops.
To resolv this I mounted the image to another dir, mv'd all the files back, and use a tmpfs in ram to copy over the convertfs files to from /sbin and it worked finally ;p
sure takes a long time for it to mv over to itself 12 gigs of data tho. |
|
Back to top |
|
|
Deathwing00 Bodhisattva
Joined: 13 Jun 2003 Posts: 4087 Location: Dresden, Germany
|
Posted: Mon Jun 14, 2004 10:51 pm Post subject: |
|
|
The command covertfs does not exist any more in the RIP CD... so this Doc is OBSOLETE/NEEDS RENEWAL.
OK, I have tweaked around and finally got how to do it:
--- Root Partition to be converted ---
1. If the partition you want to convert is the root one, then boot into your system.
2. # emerge convertfs
3. reboot the system with the gentoo LiveCD
4. mount your root partition (the one to be converted)
5. copy the following files from /mnt/gentoo/sbin/ to /sbin/:
- convertfs
- convertfs_dumb
- devclone
- devremap
- prepindex
6. unmount the partition
7. (this one is important!) # cd /sbin/
8. # convertfs /dev/hdxY current_fs target_fs
9. wait until it ends
--- Non-root partition to be converted ---
1. If the partition you want to conver is not the root one, then it's easier; boot into your system.
2. # emerge convertfs
3. make sure that the partition is not mounted
4. # convertfs /dev/hdxY current_fs target_fs
5. wait until it ends
Have fun! |
|
Back to top |
|
|
thrasher6670 Apprentice
Joined: 04 Aug 2003 Posts: 269 Location: London, Ontario
|
|
Back to top |
|
|
Deathwing00 Bodhisattva
Joined: 13 Jun 2003 Posts: 4087 Location: Dresden, Germany
|
Posted: Wed Jun 23, 2004 5:20 pm Post subject: |
|
|
Nice one indeed |
|
Back to top |
|
|
ZeroDivide Tux's lil' helper
Joined: 15 Mar 2003 Posts: 86
|
Posted: Fri Jul 09, 2004 11:02 pm Post subject: |
|
|
I'm not sure if it's just my setup, but convertfs is EXTREMELY slow.
It's taking about 3 1/2 hours to convert 1 gig of data from reiserfs to xfs.
I have DMA enabled and 1/3 of the partition is free space so I dont think its a problem with my drive.
Anyway, just thought I would post this to let people know how slow it can be before they embark on their long conversion journey |
|
Back to top |
|
|
Deathwing00 Bodhisattva
Joined: 13 Jun 2003 Posts: 4087 Location: Dresden, Germany
|
Posted: Sat Jul 10, 2004 7:54 am Post subject: |
|
|
This is very strange... it didn't take so much time for me. |
|
Back to top |
|
|
ZeroDivide Tux's lil' helper
Joined: 15 Mar 2003 Posts: 86
|
Posted: Sat Jul 10, 2004 3:31 pm Post subject: |
|
|
It looks like i jumped the gun a little bit, converting 15GB only took 6 hours. |
|
Back to top |
|
|
asph l33t
Joined: 25 Aug 2003 Posts: 741 Location: Barcelona, Spain
|
Posted: Mon Aug 23, 2004 8:09 am Post subject: |
|
|
i think that "How to convert your filesystem and lose all your data" would be more accurate.. i run it to convert from reiserfs to xfs, and after 2 secons it output "mv: comand not found" and stopped working.. after rebooting i had lost config files, binaries and a bunch of files so it won't boot!
of course i did a backup before so i have my system back again, but i don't understand how this tool is in portage, it totally sucks. You can't release a tool this dangerous if you are not 100% sure it works! I know they say that "warning, this can delete your data blah blah", but then just don't put it on portage. _________________ gentoo sex is updatedb; locate; talk; date; cd; strip; look; touch; finger; unzip; uptime; gawk; head; emerge --oneshot condom; mount; fsck; gasp; more; yes; yes; yes; more; umount; emerge -C condom; make clean; sleep |
|
Back to top |
|
|
Moloch Apprentice
Joined: 17 Mar 2003 Posts: 293 Location: Albuquerque, NM, US
|
Posted: Wed Aug 25, 2004 6:46 pm Post subject: |
|
|
Sounds like you ran it from your current live system. Idealy you want to run it from a boot CD so you aren't attempting to convert files/partitions that are in use. |
|
Back to top |
|
|
Conan Guru
Joined: 02 Nov 2004 Posts: 360
|
Posted: Tue Nov 02, 2004 1:28 am Post subject: |
|
|
I just tried doing this... and when i try to copy convertfs from /mnt/gentoo/sbin/ to /sbin, it tells me that /sbin/ is a read only file system, is there some way to fix this? |
|
Back to top |
|
|
bigredgiant1 n00b
Joined: 23 Dec 2003 Posts: 62
|
Posted: Tue Jan 04, 2005 10:55 pm Post subject: |
|
|
I think a better way to do this if you had the space would be to copy everything over to a second drive, make the new filesystems on your partitions, and copy everything back over, like so:
Code: |
<boot a live cd and mount all your partitions/drives>
# chroot /mnt/gentoo
# mount /dev/hdb1 /mnt/tmp_drive
# mkdir /mnt/tmp_drive/gentoo
# cd /
# find . -print | cpio -pamd /mnt/tmp_drive/gentoo
# umount /mnt/tmp_drive/gentoo
# exit
<this will probably take at least 10 or 15 minutes>
# umount /dev/hda3 /dev/hda5 ...
<I use reiserfs here but use whatever filesystem you desire>
# mkreiserfs /dev/hda3
# mkreiserfs /dev/hda5
<continue for all of your partitions...>
# mount /dev/hdb1 /mnt/tmp_drive
# mount /dev/hda3 /mnt/gentoo
# mount /dev/hda5 /mnt/gentoo/var
<continue as needed>
# cd /mnt/tmp_drive/gentoo
# find . -print | cpio -pamd /mnt/gentoo
<Reboot and you're done> |
_________________ Vic @ Shellsage
Dual Opteron 242 -- AMD64 2.6.16 Development Sources
CFLAGS="-march=k8 -O3 -pipe" MAKEOPTS="-j3" |
|
Back to top |
|
|
rvkasper n00b
Joined: 17 Mar 2005 Posts: 1 Location: Denver, CO USA
|
Posted: Thu Mar 17, 2005 4:47 pm Post subject: I'm an idiot |
|
|
Hi all,
I decided to convert a 200 gig partition from reiserfs to ext2 so that it will work when I move my Fedora Core 2 OS over to FreeBSD - to facilitate this I compiled convertfs from source and am running it from my current OS installation. 4 days later it is still going, running devremap - it is currently here:
Relocating block group at 2899145...
Any thoughts on how far it has left to go - is there some output from a command (like HDParm) that would help me figure that out?
Also, would booting from this gentoo livecd speed things up at all? This is not my root partition I am converting - simplay a data partition - my root partition is ext3.
- Ryan _________________ Taco taco, burrrito burrito! |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Thu Sep 29, 2005 12:58 pm Post subject: |
|
|
Two issues with convertfs:
1. You need a lot of free space on the partition you want to convert.
convertfs uses 'mv' to move all directories (including subdirs and files) from the root tree to the image. When moving directories, 'mv' will delete the source files only after the whole thing was copied, so you need at least as much free space as the space consumed by the biggest directory (including subdirs and files) on your partition. You can find out that size using 'du'.
2. Relocation of blocks is damn slow.
After the image was completely created, you have your old filesystem with a single file in it (image of the new filesystem), and in that image there are all your files. Now, for the image to replace your original filesystem, convertfs has to move the blocks of the image file onto the appropriate positions on your hard disk.
Think of your partition now as a really fragmented system and you're trying to use Windows defrag to solve the system - every block of every single file has to be moved to a different location (and that location might already be occupied by other blocks).
So in short, there is a lot of shuffling around involved, and in the current convertfs, this is not solved very efficiently, so depending on how much block relocations have to be done, this will take AGES. And I really mean AGES - I'm trying to convertfs a 80GB partition from ext3 to xfs right now and after one day, the progress is at 1GB out of 80GB. So with any luck I only have to wait 79 more days for it to get finished.
The problem lies within devremap.c, in the function find_cross_block, which wastes a lot of time trying to find the correct physp location. The CPU has to do an awful lot of work, disk activity is extremely low (looks like one block write every 20 seconds).
I don't fully understand what devremap.c is trying to do here (there does not seem to be any documentation, grrr), but this can be done faster in any case. |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Thu Sep 29, 2005 7:33 pm Post subject: |
|
|
filed a bug report for this: sys-fs/convertfs eats 100% CPU for days on end
comments are welcome...
EDIT:
attached a patch to the bug report, which fixed the 100% CPU / very slow progress problem for me. conversion of 80GB partiton ext3 -> xfs succeeded . Still, this is highly dangerous software and it has other bugs as well. But if you're already stuck in the 'relocating blocks' part of convertfs, the patch might help you. It's reasonably safe to ctrl+c out of the relocating (while there is nothing being written on the hard disk) and resume with the patched program.
Oh, and make sure you compile it with -march=your_cpu_type and -O2. It'll be really slow otherwise. And don't use -O3, for some reason that slows down the process as well. |
|
Back to top |
|
|
dj015 n00b
Joined: 05 Jul 2004 Posts: 11
|
Posted: Tue Jan 17, 2006 6:44 am Post subject: |
|
|
For those people that, like me, found this forum too late:
1. In the "Relocating block group at X" messages, X is the filesystem block number, not a kilobyte number. So if you have a filesystem with 4kB blocks, multiply X by 4 to get the number of kilobytes.
2. The reason it is slow is the poor block relocation algorithm (you can't code a triply nested loop, and then use "gcc -O3" to make it fast: compilers never reduce the algorithm's order of complexity...).
3. convertfs is very robust - it survived about 5 Ctrl+C's and a power failure.
4. The "very ugly patch" works like a charm. It stripped down the conversion time for my 120 GB filesystem from 11 days to 6 hours. I did e2fsck afterwards and it found no errors, my files were all there, and I opened a couple of them and they were fine. Thank you Andreas!
5. I am never using it again. |
|
Back to top |
|
|
dajja n00b
Joined: 11 May 2006 Posts: 15 Location: Sweden
|
Posted: Sat May 20, 2006 12:32 pm Post subject: |
|
|
I have recently successfully converted an ext3 partition to reiserfs (no corruption reported by fsck)
But I am (still) somewhat suspicious if it will work in the long run (I will probably notice if it doesn't )---is fsck completely reliable in reporting a possible corruption/error in a filesystem? Are there other tools to consider? |
|
Back to top |
|
|
|