HOWTO: Easily make a full system backup (stage4)

Message

BlinkEye · Post by **BlinkEye** » Fri Apr 01, 2005 2:37 pm

Sade wrote:just wondering: is the ccache also stored in the archive? it would be nice if u had the choise.

well, i gave some hints under "2. How to customise". i don't know where you put your ccache. check the variables, if it is a subfolder of a folder listed in the exclude variables add it to $default_include_list. if not put it either in $default_exclude_list or $custom_exclude_list (i suggest in $custom_exclude_list).

Sade · Post by **Sade** » Fri Apr 01, 2005 2:50 pm

i will try, i think i can complete that puzzle.

d_adams · Post by **d_adams** » Sun Apr 03, 2005 4:20 am

Just tried this one out, and I must say, very nice work indeed. One of my kid's boxes had to be reloaded, so I used my backup to install gentoo on their box, instead of suse or windows, as it had both and was getting pretty messed up. I used the 1.2 version initially and got the console display error. I'm doing the 1.4 version now, and it's a lot better than doing a stage 1 or 3 install on a athlon xp 1900. Just need to create a user account for her and I'll be done. I set the backup file on my internal ftp, and used links from the live cd to get the file, and then untarred it. Very quick way to install gentoo.

Thanks a million for this one.

DNL · Post by **DNL** » Mon Apr 04, 2005 12:35 am

Just want to drop in and add my thanks... Getting ready to try some stupid things for the sake of my own stupidity, and did a quick search for backup and found this. Excellent, I was hoping I wouldn't have to write my own. Thanks a lot for sharing this. Gotta love open source.

Jerri · Post by **Jerri** » Mon Apr 04, 2005 1:47 am

Sade wrote:when i select copy and paste, i get a lot of spaces in my file, and i don't know how not to select them, or smartly remove the spaces.

cat file | sed '/^$/d'

check this out for more tricks...

DNL · Post by **DNL** » Mon Apr 04, 2005 11:03 am

Well, unless I'm missing something, restoring this backup worked perfectly. Thanks again!

Highly recommend n00bs run this before doing anything they are unsure of. Could have used this before my first emerge --deep --newuse world...

BlinkEye · Post by **BlinkEye** » Mon Apr 04, 2005 5:54 pm

thanks for the feedbacks. i'm glad it's working for you guys too ...

Uncle_Psychosis · Post by **Uncle_Psychosis** » Mon Apr 04, 2005 8:08 pm

Hi

This looks like a great script, thanks!

Could someone please help me out a little- I'm a bit confused as to how to use the backup once I've got it. What I want to do is use the stage4 to copy my system onto a new HDD. What do I need to do?

Here is what I *think* needs to be done:

boot from live cd into new hard drive, as if installing from scratch
fdisk accordingly
make filesystems
activate swap
mount /mnt/gentoo and /mnt/gentoo/boot in the right places
cd /mnt/gentoo
extract stage4
install portage snapshot [as per instructions in handbook]??
chroot
emerge sync
configure kernel??
configure /etc/fstab accordingly
configure Grub
Install system loggers and tools??
Create passwords and users

Have I missed anything or done anything unnecessary? Or do I just follow the handbook instructions, but replace their stage tarball with my own? Sorry for the silly questions!

Thanks

Sam

Jerri · Post by **Jerri** » Mon Apr 04, 2005 11:31 pm

configure kernel??
configure /etc/fstab accordingly
configure Grub
Install system loggers and tools??
Create passwords and users

if your backing up a working install, these should already be taken care of. You might have to update your MBR if it was damaged... but everything else whould have been configured before you backed it up.

Basically, this whole process takes a snapshot of your system, as is (mind you, everything that is created when you boot, like sys/* and proc/ are excluded).

Uncle_Psychosis · Post by **Uncle_Psychosis** » Mon Apr 04, 2005 11:37 pm

Ok, thanks. I just tried it, and it complained that my .bz2 file was corrupted. Boogar. Will try again tommorrow.

Thanks

Sam

mascanho · Post by **mascanho** » Tue Apr 05, 2005 1:10 am

Damn, what a usefull script very good indeed, tanks man it realy saves a lot of work and time

, tested it here and it works great

Asmdroid · Post by **Asmdroid** » Fri Apr 08, 2005 1:53 pm

Hi, I have two systems a laptop and a pentium 4 driven desktop workstation. I'm wondering if the CFLAGs I used on my main Gentoo workstation (march=pentium4) are p4 optimized CFLAGS and if this means they will traumatise (or not) my i686 Celeron laptop system due to both processors being compatible with the same architecture somewhat?

Asmdroid · Post by **Asmdroid** » Fri Apr 08, 2005 1:58 pm

Uncle_Psychosis wrote:Ok, thanks. I just tried it, and it complained that my .bz2 file was corrupted. Boogar. Will try again tommorrow.

Thanks

Sam

Strange, me too, I might try tar.gz as space isn't an issue on my external hard drive. I'm guessing that bz2 doesn't find large achives attractive? Perhaps tar.gz will perform better. Integrity check failure.

"Unexpected EOF (End Of File)"

BlinkEye · Post by **BlinkEye** » Fri Apr 08, 2005 2:21 pm

Asmdroid wrote:Hi, I have two systems a laptop and a pentium 4 driven desktop workstation. I'm wondering if the CFLAGs I used on my main Gentoo workstation (march=pentium4) are p4 optimized CFLAGS and if this means they will triaumatise (or not) my i686 Celeron laptop system due to both processors being compatible with the same architecture somewhat?

i'm afraid but that won't work.

/etc/make.conf wrote: # Host and optimization settings
# ==============================
#
# For optimal performance, enable a CFLAGS setting appropriate for your CPU.
#
# Please note that if you experience strange issues with a package, it may be
# due to gcc's optimizations interacting in a strange way. Please test the
# package (and in some cases the libraries it uses) at default optimizations
# before reporting errors to developers.
#
# -mcpu=<cpu-type> means optimize code for the particular type of CPU without
# breaking compatibility with other CPUs.
#
# -march=<cpu-type> means to take full advantage of the ABI and instructions
# for the particular CPU; this will break compatibility with older CPUs (for
# example, -march=athlon-xp code will not run on a regular Athlon, and
# -march=i686 code will not run on a Pentium Classic.

BlinkEye · Post by **BlinkEye** » Fri Apr 08, 2005 2:25 pm

Asmdroid wrote:
Uncle_Psychosis wrote:Ok, thanks. I just tried it, and it complained that my .bz2 file was corrupted. Boogar. Will try again tommorrow.

Thanks

Sam
Strange, me too, I might try tar.gz as space isn't an issue on my external hard drive. I'm guessing that bz2 doesn't find large achives attractive? Perhaps tar.gz will perform better. Integrity check failure.

"Unexpected EOF (End Of File)"

could you guys specify your problem? how big is your stage4? i never had a tar.bz2 failing on me, although i've started to use tar.gz because of the time gaining. there is a limit to the hole size of the archive (filesystem dependent) - but the creation for the stage4 would stop while creating the tar, so, this isn't the problem

Asmdroid · Post by **Asmdroid** » Fri Apr 08, 2005 2:31 pm

BlinkEye wrote: stage4? i never had a tar.bz2 failing on me, although i've started to use tar.gz because of the time gaining. there is a limit to the hole size of the archive (filesystem dependent) - but the creation for the stage4 would stop while creating the tar, so, this isn't the problem

The file that I made was just over 1GB (1.1). I originally stored it to the default directory in the script then transferred across to my USB hard-drive, this took a fair amount of time after continuously sending parts then caching. This could be a problem with my hardware, I'm not sure. The file did include a few useless directories also, CCACHE for example, which isn't appropriate.

BlinkEye · Post by **BlinkEye** » Fri Apr 08, 2005 2:43 pm

do you still have the tar.bz2? could could you experiment further (like extracting ? it doesn't matter if you've inlcuded the ccache folder or not, a tar.bz2 is a tar.bz2 and MUST work.

EDIT: i'm just remembering that i had a lot of errors lately extracting some zipped files, so, did it just break or was it a warning/error at the end of the extracting? if possible post the error message

Mr Faber · Post by **Mr Faber** » Fri Apr 08, 2005 8:56 pm

Thanks, this is a really great and easy to use script. It makes life much easier

Maybe an option for incrementell backups is usefull for future versions.

cu
Mr Faber

devils · Post by **devils** » Sat Apr 09, 2005 12:28 pm

Thanks alot man for doing this very cool script.
Exactly what i have been looking for...

/devil

Uncle_Psychosis · Post by **Uncle_Psychosis** » Sat Apr 09, 2005 5:14 pm

BlinkEye wrote:do you still have the tar.bz2? could could you experiment further (like extracting ? it doesn't matter if you've inlcuded the ccache folder or not, a tar.bz2 is a tar.bz2 and MUST work.

EDIT: i'm just remembering that i had a lot of errors lately extracting some zipped files, so, did it just break or was it a warning/error at the end of the extracting? if possible post the error message

I'm afraid I binned the file so I can't be much help. It was about 3.5Gb because I'd also backed up my /home.

I've managed to restore my system onto a new hard drive using this, but I've ran into a few problems. For instance, I was getting a "Error opening terminal: Eterm." error everytime I tried to run nano, so I tried to fix this by re-emerging ncurses (which seems to be a fix for this sort of problem). However, ncurses fails with:

Code: Select all

Could not run/locate "g++"
make[1]: *** [../obj_s/cursesf.o] Error 1
make[1]: *** Waiting for unfinished jobs....
Could not run/locate "g++"
make[1]: *** [../obj_s/cursesm.o] Error 1
make[1]: Leaving directory `/var/tmp/portage/ncurses-5.4-r5/work/ncurses-5.4/c++'
make: *** [all] Error 2
+ diefunc src_compile 84 2 'make failed'
+ local funcname=src_compile lineno=84 exitcode=2
+ shift 3
+ echo

+ echo '!!! ERROR: sys-libs/ncurses-5.4-r5 failed.'
!!! ERROR: sys-libs/ncurses-5.4-r5 failed.
+ echo '!!! Function src_compile, Line 84, Exitcode 2'
!!! Function src_compile, Line 84, Exitcode 2
+ echo '!!! make failed'
!!! make failed

Any ideas? I saw when I unpacked my stage 4 that it complained about not being able to hardlink a lot of files (don't ask which, it was scrolling way too fast!). Could this be the source of the problem? Do I need to just do an emerge -uD system/world to fix it?

Cheers

Sam [/code]

feld · Post by **feld** » Sat Apr 09, 2005 11:43 pm

thank you SO much for this script. I just backed up, wiped my system, and tested the restore. IT WORKED FLAWLESSLY.

I really REALLY appreciate all the hard work you put into this backup script. Finally I have something easy and reliable enough to make me want to backup nearly every day

(ok ok, i'll keep it to once a week!)

thanks again!

-Feld

BlinkEye · Post by **BlinkEye** » Sun Apr 10, 2005 7:04 am

Uncle_Psychosis wrote: I've managed to restore my system onto a new hard drive using this, but I've ran into a few problems. For instance, I was getting a "Error opening terminal: Eterm." error everytime I tried to run nano, so I tried to fix this by re-emerging ncurses (which seems to be a fix for this sort of problem). However, ncurses fails with:
Code: Select all
Could not run/locate "g++"
make[1]: *** [../obj_s/cursesf.o] Error 1
make[1]: *** Waiting for unfinished jobs....
Could not run/locate "g++"
make[1]: *** [../obj_s/cursesm.o] Error 1
make[1]: Leaving directory `/var/tmp/portage/ncurses-5.4-r5/work/ncurses-5.4/c++'
make: *** [all] Error 2
+ diefunc src_compile 84 2 'make failed'
+ local funcname=src_compile lineno=84 exitcode=2
+ shift 3
+ echo

+ echo '!!! ERROR: sys-libs/ncurses-5.4-r5 failed.'
!!! ERROR: sys-libs/ncurses-5.4-r5 failed.
+ echo '!!! Function src_compile, Line 84, Exitcode 2'
!!! Function src_compile, Line 84, Exitcode 2
+ echo '!!! make failed'
!!! make failed
Any ideas? I saw when I unpacked my stage 4 that it complained about not being able to hardlink a lot of files (don't ask which, it was scrolling way too fast!). Could this be the source of the problem? Do I need to just do an emerge -uD system/world to fix it?

Cheers

Sam

ncurses complains about not being able to run/locate the C++ compiler, which is bad. so i guess you can't emerge anything. i don't know how this happened but could it be due to your personal setup? have you had any hardlinks at all? what does

Code: Select all

# emerge --info
# gcc -info

show?

the only option for hardlinks (if you have any) i saw in tar is

Code: Select all

--check-links
              warn if number of hard links to the file on the filesystem mismatch the number of links recorded in the archive

so, you could change

Code: Select all

# options for the tar command 
 tarOptions="--absolute-names --preserve-permissions --totals --ignore-failed-read --verbose --file"

to

Code: Select all

# options for the tar command 
 tarOptions="--absolute-names --preserve-permissions --totals --check-links --ignore-failed-read --verbose --file"

but you need to do that only if you're working with hardlinks and i don't know if it will solve your problem or not. you're the first complaining about this.

Uncle_Psychosis · Post by **Uncle_Psychosis** » Sun Apr 10, 2005 10:31 am

BlinkEye wrote: but you need to do that only if you're working with hardlinks and i don't know if it will solve your problem or not. you're the first complaining about this.

I don't know. I know that I have personally never created any hardlinks...

Of course, the reason I'm doing this restore in this first place is because my old hard drive is on the way out (system just freezes and the logs show a whole load of i/o errors). It might well be that that is the cause of the problem. I did manage to fix this problem by doing a

#emerge --usepkg gcc
#emerge ncurses

Other than that, everything seems to be in tip-top condition. Thanks very much for the script---you've saved me hours of work!

Sam

aschueler · Post by **aschueler** » Sun Apr 10, 2005 5:49 pm

How do you recombine the split bzipped tar files? man split doesn't seem to tell me unless I am missing something.

Thanks for a great program that I hope I will never need!

BlinkEye · Post by **BlinkEye** » Sun Apr 10, 2005 7:49 pm

Code: Select all

cat amd64-stage4-10.04.2005-minimal.tar.gz_* > amd64-stage4-10.04.2005-minimal.tar.gz

or

Code: Select all

cat amd64-stage4-10.04.2005-minimal.tar.bz2_* > amd64-stage4-10.04.2005-minimal.tar.bz2

of course you may enumerate your splitted files instead of using an asterix

HOWTO: Easily make a full system backup (stage4)

Compilation

Re: Compilation