HOWTO: Easily make a full system backup (stage4)

[Ejunkie]

actually hey BlinkEye, i wonder if there's a way to set it up to save your kernal .config file even if you don't save your sources. that if you have to dokernel recomplies later on you don't have to try to remember all the settings you had. i'm sure it's doable, i just don't know enough about bash scripting to try to customize it in

put it under "custom_include_list"

like so

Code: Select all

custom_include_list="
/usr/src/linux/.config
or
/usr/src/linux-`uname -r`/.config
"

[-valheru-]

To be on the safe side I am repeating the restore process and will advise. Portgage is syncing right now.

EDIT: I noticed before and I'm sure I will again after the portage syncs that these commands:

10. mkdir /mnt/gentoo/proc
11. mount -t proc none /mnt/gentoo/proc
12. mkdir /mnt/gentoo/dev

Don't do anything because the proc and dev directories are already there. Does this only apply for someone who does the interactive, FULL, backup?

Thanks

[BlinkEye]

actually hey BlinkEye, i wonder if there's a way to set it up to save your kernal .config file even if you don't save your sources. that if you have to dokernel recomplies later on you don't have to try to remember all the settings you had. i'm sure it's doable, i just don't know enough about bash scripting to try to customize it in

put it under "custom_include_list"

like so

Code: Select all

custom_include_list="
/usr/src/linux/.config
or
/usr/src/linux-`uname -r`/.config
"

hmm, i had this very line in the custom_include_list myself. must have dropt out while customizing the script.

To be on the safe side I am repeating the restore process and will advise. Portgage is syncing right now.

EDIT: I noticed before and I'm sure I will again after the portage syncs that these commands:

10. mkdir /mnt/gentoo/proc
11. mount -t proc none /mnt/gentoo/proc
12. mkdir /mnt/gentoo/dev
13. mount -o bind /dev /mnt/gentoo/dev

Don't do anything because the proc and dev directories are already there. Does this only apply for someone who does the interactive, FULL, backup?

Thanks

well, you mount a partition onto /mnt/gentoo in step 6. if you have a /dev and /proc directory still remaining on that partition you do not need to mkdir them (just to be sure - you don't extract the stage4 onto a partition/harddisk where you had a running (gentoo) linux installation, do you? if so, "delete" all the files by recreating a filesystem for example).

[-valheru-]

This was installed upon a freshly formated hard drive. I had guessed it was due to the my doing a full interactive backup. Since I used fdisk to delete and then re-create the filesystems, those directories should not have existed correct? Does doing the full, interactive backup save those directories? Thanks again.

EDIT: this happened 2x actually. Any ideas?

[Ejunkie]

This was installed upon a freshly formated hard drive. I had guessed it was due to the my doing a full interactive backup. Since I used fdisk to delete and then re-create the filesystems, those directories should not have existed correct? Does doing the full, interactive backup save those directories? Thanks again.

EDIT: this happened 2x actually. Any ideas?

are the maps /mnt/gentoo/proc and /mnt/gentoo/dev/ are those maps empty if thay do just mount then else clear the partition again.
or do "dd if=/dev/zero of=/dev/hda2" ow where else you want to install gentoo.

[-valheru-]

The hard drive the backup is being restored to is a freshly partitioned file system so there is not an existing proc or dev directory. I'll try "dd if=/dev/zero of=/dev/hdaX" and see what happens. I guess that clears or formats the drive?

I'll let you know what happens. I just thought the backup script backed up my existing proc and dev if I did the full, interactive backup telling it to backup everything. No biggie.

EDIT: I reviewed the script. Seems it does backup the proc and dev folders. So no problem then. Thanks. See below:

Code: Select all

# these files/directories are always excluded. don't add trailing slashes. 
# don't touch it unless you know what you are doing! 
# /var/db and /var/cache/edb are intentionally added here. they are listed 
# in $default_include_folders 
default_exclude_list=" 
/dev 
/lost+found 
/mnt 
/proc 
/sys 
/tmp 
/usr/portage 
/usr/src 
/var/log 
/var/tmp 
/var/db 
/var/cache/edb 
$stage4Location 
`echo $CCACHE_DIR`" 

# files/devices/folders, which need to be backed up (preserve folder structure). 
# don't touch it unless you know what you are doing! no recursive backup of folders. 
# use $default_include_folders instead. 
default_include_files=" 
/dev/null 
/dev/console 
/home 
/mnt 
/proc 
/sys 
/tmp 
/usr/portage 
/usr/src 
/var/log/emerge.log" 

[BlinkEye]

i made some minor changes to the howto and the script itself. i tried to include all the tips given since the last time. i inserted a new link at the beginning of the first post - i've put the howto into my wiki as it's far easier to maintain and to read.

[-valheru-]

Cool. What is the link to your wiki?

EDIT: Nevermind, followed the link in your signature to the other forum which has a link to your WIKI. THANKS

EDIT: YOU PLACED my name in your WIKI-------WHHOOOOOHOOOOOOOOO !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[paulb787]

i dont understand steps 10 and 11 under restore? are these steps nessacary if u have done a full(everything) backup. because i have /proc direstory?

10. mount -t proc none /mnt/gentoo/proc
11. mount -o bind /dev /mnt/gentoo/dev

[-valheru-]

well you still have to MOUNT them, just not MKDIR them.

[b8zs]

If I have /dev/md1 as /
and /dev/md0 as /boot

Would I be able to use this script to backup my machine?

I'm thinking just make it copy over my raidtab file, but will I run into any snags with a software raid0?

[Ejunkie]

If I have /dev/md1 as /
and /dev/md0 as /boot

Would I be able to use this script to backup my machine?

I'm thinking just make it copy over my raidtab file, but will I run into any snags with a software raid0?

yes for so var i know you can use it.

[zigver]

Not really a problem with the script but has anyone noticed that 'uname -r' for the latest gentoo-sources doesn't report what is in /usr/src? It's easily fixed with a link but an odd issue. Has this always been the case and I just didn't notice?

[satanskin]

Ok, not sure if this has been mentioned before or not but it seems that no matter what I do, the integrity check at the end always fails. I get this at the end:

Code: Select all

* stage4 is done
* umounting boot
* Checking integrity
* Integrity check failed. Re-run the script and check your hardware.
thor satanskin # 

I'm not sure why it would fail so many times when nothing should be wrong. Any ideas?

[BlinkEye]

this integrity check happens to fail for me too sometimes. i'm not yet sure why this happens - because i'm almost confident the stage4 is intact anyway.

do you still have suche a "broken" stage4 ? please run the following command and post the output:

Code: Select all

gzip --test  stage4.tar.gz

or

Code: Select all

bzip2 --test  stage4.tar.bz2

[satanskin]

Yes, it still says that it is failing. In regards to the command you suggested:

Code: Select all

thor satanskin # bzip2 --test /mnt/tyr/stage4/thor-stage4-2005.11.23-custom.tar.bz2_a
bzip2: /mnt/tyr/stage4/thor-stage4-2005.11.23-custom.tar.bz2_a: data integrity (CRC) error in data

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

thor satanskin # 

[BlinkEye]

ah, you use a splitted stage4. could you please post the output of:

Code: Select all

cat stage4.bz2_* | bzip2 --test

[satanskin]

Code: Select all

thor satanskin # cat /mnt/tyr/stage4/thor-stage4-2005.11.23-custom.tar.bz2_* | bzip2 --test
bzip2: (stdin): data integrity (CRC) error in data

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

thor satanskin # 

[BlinkEye]

ok, I think this really is a tar/gzip/bzip2 (or for that matter a hardware) issue and AFAIK has nothing to do with my script.

http://www.gzip.org/#faq1
http://mail-index.netbsd.org/current-us ... /0012.html (very old)

I suggest you try to contact a tar/bzip2 users list, maybe you find the problem. did you ever extract such a faulty stage4? have you tried on a different box?

[satanskin]

no, haven't tried extracting it yet, but perhaps i can try it in a few days. Thanks for trying though man and thanks for the nice script!

[-valheru-]

I too get errors trying to bz2 with a file over 2.5 gigs. File intergrity error.

gz is fine though.

[carpman]

I too get errors trying to bz2 with a file over 2.5 gigs. File intergrity error.

gz is fine though.

i have a 5gb backup that is fine!

[bingobob]

This script is great. One thing though. I would like to run it each week from a cron script. How can i make it non-interactive?

[amaroc]

BlinkEye,

great script, really easy to use.

I'm doing the minimal-tgz-backup to my Buffalo-LinkStation via WLAN. The ftp-backupdrive is mounted via lufs

Code: Select all

lufis fs=ftpfs,host=my_host,username=foo,password=bar,ftpactive /mnt/backups/ 

and it works flawlessly.

Thanks!

PS: I did not modify the script to handle /home, /usr/src, etc. according to my needs. Such directories will be backuped using krusader's built-in ftp- and archive-capabilities.

[andrewd18]

I'm getting an issue - it can't pass the integrity check...

Code: Select all

StubbornAesthetics ~ # ./mkstage4.sh


Backup script v3.6
==================

What do you want to do? (Use CONTROL-C to abort)

Fast (tar.gz):
 (1) Minimal backup
 (2) Interactive backup


Best (tar.bz2):
 (3) Minimal backup
 (4) Interactive backup

Please enter your option: 3

* mounting boot

Do you want to continue? (y/n) y
* creating directory /media/IOMEGA_HDD/Backups/stage4

* Please wait while the stage4 is being created.

sh: line 1:  2708 Broken pipe             /usr/bin/find /* -path /dev -prune -o -path /lost+found -prune -o -path /mnt -prune -o -path /media -prune -o -path /proc -prune -o -path /sys -prune -o -path /tmp -prune -o -path /usr/portage -prune -o -path /usr/src -prune -o -path /var/log -prune -o -path /var/tmp -prune -o -path /var/db -prune -o -path /var/cache/edb -prune -o -path /media/IOMEGA_HDD/Backups/stage4 -prune -o -path /home/andrew -prune -o -path /home/lost+found -prune -o -path /usr/src/linux-2.6.14-gentoo-r2 -prune -o -print
sh: line 1:  2663 Done(141)               ( /usr/bin/find /var/db; echo /usr/src/linux-2.6.14-gentoo-r2/.config; echo /var/log/emerge.log; echo /usr/src; echo /usr/portage; echo /tmp; echo /sys; echo /proc; echo /media; echo /mnt/floppy/.keep; echo /mnt/cdrom/.keep; echo /mnt/.keep; echo /mnt; echo /home; echo /dev/console; echo /dev/null; /usr/bin/find /* -path /dev -prune -o -path /lost+found -prune -o -path /mnt -prune -o -path /media -prune -o -path /proc -prune -o -path /sys -prune -o -path /tmp -prune -o -path /usr/portage -prune -o -path /usr/src -prune -o -path /var/log -prune -o -path /var/tmp -prune -o -path /var/db -prune -o -path /var/cache/edb -prune -o -path /media/IOMEGA_HDD/Backups/stage4 -prune -o -path /home/andrew -prune -o -path /home/lost+found -prune -o -path /usr/src/linux-2.6.14-gentoo-r2 -prune -o -print )
      2665 Broken pipe             | /bin/tar --bzip2 --preserve-permissions --create --absolute-names --totals --ignore-failed-read --file /media/IOMEGA_HDD/Backups/stage4/StubbornAesthetics-stage4-2005.12.04-minimal.tar.bz2 --no-recursion -T -

* stage4 is done
* umounting boot
* Checking integrity
* Integrity check failed. Re-run the script and check your hardware.

Suggestions? Same problem funkoolow had, but there was no answer posted to his inquiry.

~~ Andrew D.