Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Easily make a full system backup (stage4)
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3 ... 16, 17, 18  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Wed Mar 23, 2005 10:07 am    Post subject: HOWTO: Easily make a full system backup (stage4) Reply with quote

Please go here for the most up-to-date version of this howto. i put this howto into my wiki as it's far easier to maintain and a lot more clearly laid out.






INDEX
    0. What's this all about
    1. How it works
    2. Features
    3. Portability/Prerequisites
    4. How to customise
    5. The result
    6. The script
    7. Download
    8. Run it
    9. Restore
    10. Changelog

0. What's this all about
i want to backup my system quickly, easily and with minimum effort. that's why i wrote this script. the resulting backup file (.tar.bz2 or .tar.gz) is called a stage4. you're probably a gentoo user and hence know the three stages gentoo uses. this backup will be a stage4, meaning, it will be a full system backup of your current system. I got inspired by this thread. if you want to know how it works, continue reading, if not, there's no reason to read all through this thread - jump to Section "8. Run it".

1. How it works
it's a bash script which uses only "standard" tools, namely "tar" and if you intend to burn the stage4 on a cd you'll probably use "split" too. this means this script works out of the box. it has been modified a lot and i now tried to detect any error/misconfiguration a user might do (wrong $PATH, wrong commands, wrong exclude list, overwriting existing files ...).
after checking everything's all set up the script will provide you with four options:
Code:
Backup script v3.5
==================

What do you want to do? (Use CONTROL-C to abort)

Fast (tar.gz):
 (1) Minimal backup
 (2) Interactive backup


Best (tar.bz2):
 (3) Minimal backup
 (4) Interactive backup

Please enter your option:


Minimal backup
this option allows you to quickly build a stage4.tar.bz2/stage4.tar.gz without any further fuss, which means
1. it will exclude (i.e. NOT backup) any files/directories listed in the $custom_list variable
2. result in a minimal stage4.

Interactive backup
this option will ask you for any file/directory listed in the custom_list variable if you want it to backup. this might result in the same stage4.tar.bz2/stage4.tar.gz if you answer all question with "no" or in a considerably bigger stage4.

Parameters
the script only shows error on stdout per default. if you want to backup in verbose mode and watch the files processed by tar call the script like:
Code:
./mkstage4.sh --verbose

or
Code:
./mkstage4.sh -v

if you intend to backup your script on a cd/dvd with a predefined size call the script like:
Code:
./mkstage4.sh --split

or
Code:
./mkstage4.sh -s

don't forget to adjust the $split_options variable, i.e. to set the desired chunk size. default is 685MB.

of course you may combine command line parameters in any way.

2. Features
* works right out of the box.
* checks if all needed tools exist.
* checks that any file/folder listed exists (no spelling errors).
* names backups according to its hostname and the date it was created -> unique, meaningful file name.
* prevents overwriting a backup file created on the same day.
* does an integrity check after the stage4 has been created.
* allows fine-tuning and easy change.
* verbose mode.
* command line option --split for splitting the tarball on the fly.

3. Portability/Prerequisites
Against some misconceptions you may use your stage4 for a box with a different CPU as well. If - and only if - you plan to use your stage4 for another CPU type (not just another box with the same CPU), you must check your /etc/make.conf for the CFLAGS entry:
Code:
CFLAGS="-Os -mcpu=athlon -funroll-loops -pipe"

if your CFLAGS looks like this:
Code:
CFLAGS="-Os -march=athlon -funroll-loops -pipe"

you won't be able to use your stage4 for a box with a different CPU type. "march" breaks compatibility, "mcpu" doesn't. there's an easy workaround: replace "march" with "mcpu" and recompile your hole system:
Code:
emerge -e world

i created a stage4 from my laptop (CFLAGS="-Os -mcpu=pentium4 -funroll-loops -pipe"), extracted it on a usb harddisk and am able to boot my amd64 machine. applications work flawlessly.

vmware users, please read this for restore.

4. How to customise
as mentioned above there are several variables to customise. the script itself has a few comments which should get you started. you might want to change the location where the stage4.tar.bz2 is put (currently /mnt/backups/stage4), or you might not like the filename (hostname-stage4-datum.tar.bz2), or you might think that the kernel sources is part of a minimal system backup and hence you won't exclude it (and so on) ...

$default_exclude_list: put any file/directory in here which is never needed nor wanted for a minimal full system backup.
$default_exclude_pattern: exclude patterns for the $default_include_folders/files.
$default_include_files: files/folder which are needed for a minimal working system. don't add folders which should be backed up recursively. folders added here provide solely the folder structure.
$default_include_folders: folders which need to be backed-up recursively for a minimal working system.

$custom_include_list: directories which aren't needed but which may be desirable to be backed-up too interactively (like /home or /usr/src/). of course opinions about such files/directories differ, so suit yourself.
$custom_exclude_list: files/folders which are subfolders of a folder listed in $custom_include_list which should NOT be backed up.
$custom_exclude_pattern: exclude patterns for the $custom_include_list.

5. The result
the result is IMPRESSIVE.
stage4.tar.gz: you'll get a stage4.tar.gz which will take about half the time it does for a bzip2 stage4. for my laptop it takes about 20 minutes to create the stage4, my amd64 with a software raid0 needs 8 minutes. unpacking takes about 2/3 of the time it does for a bzip2 stage4. the stage4.tar.gz will be about 10% bigger than a stage4.tar.bzip2. it's your choice!
stage4.tar.bz2: you'll get a stage4.tar.bz2 (for me it's about 800MB from 3.5GB of data) which will be a fully working backup of your system. on my ibm laptop it takes about 50 minutes to create the bz2 stage4.

i recall all those hours i backup-ed my windows installation. the record was 7CD's for a full installation (inkl. some software you need for the daily use) - not bootable :twisted:.


6. The Script
Code:
#!/bin/bash
# Backup script for Gentoo Linux
# Copyright Reto Glauser aka Blinkeye
# Distributed under the terms of the GNU General Public License v2
# Mailto: stage4 at blinkeye dot ch
# Forum post: http://forums.gentoo.org/viewtopic-t-312817.html
# Date: 2005-06-30

version=v3.5
basename=`basename $0`

find=/usr/bin/find
tar=/bin/tar

# these are the commands we actually need for the backup
command_list=(cut date echo $find grep hostname mount sh split $tar umount uname which)

# verify that each command we use exists. if one can't be found use $PATH and make a suggestion if possible.
for command in ${command_list[@]}; do
   if [ ! -x "`which $command 2>&1`" ]; then
      echo -e "\nERROR: $command not found! "
      base=`basename $command`
       if [ "`which $base 2>&1 | grep "no \`basename $command\` in"`" != "" ]; then
          echo -e "ERROR: $base is not in your \$PATH."
       fi
       exit -1
   fi
done

help="\nUsage:\n\nsh `basename $0` [[-v]|[--verbose]] [[-s]|[--split]] \n\nTo run the script NOT in verbose mode comes in handy if you want to see only the errors that occur during the backup.\n"

# Defaults to creating one tarball

tar_output="--file"

# split command
split_options="--suffix-length=1 --bytes=685m"

# options for the tar command
tarOptions=" --preserve-permissions --create --absolute-names --totals --ignore-failed-read"

# where to put the stage4
stage4Location=/mnt/backups/stage4

# name prefix
stage4prefix=`hostname`-stage4-`date +\%Y.\%m.\%d`

# patterns which should not be backed up (like iso files).
# example: default_exclude_pattern="*.iso *.divx"
# These pattern count only for files NOT listed in the $custom_include_list.
default_exclude_pattern=""

# these files/directories are always excluded. don't add trailing slashes.
# don't touch it unless you know what you are doing!
# /var/db and /var/cache/edb are intentionally added here. they are listed
# in $default_include_folders
default_exclude_list="
/dev
/lost+found
/mnt
/proc
/sys
/tmp
/usr/portage
/usr/src
/var/log
/var/tmp
/var/db
/var/cache/edb
$stage4Location
`echo $CCACHE_DIR`"

# files/devices/folders, which need to be backed up (preserve folder structure).
# don't touch it unless you know what you are doing! no recursive backup of folders.
# use $default_include_folders instead.
default_include_files="
/dev/null
/dev/console
/home
/mnt
/proc
/sys
/tmp
/usr/portage
/usr/src
/var/log/emerge.log"

# folders, which need to be backed up recursively on every backup.
# don't touch it unless you know what you are doing! the reason for this
# variable is that some users add /var to the $default_exclude_list. here
# we ensure that portage's memory is backed up in any case.
default_include_folders="
/var/db"

# IMPORTANT: A minimal backup will EXCLUDE files/folders listed here. A custom backup will
# include/exclude these files/folders depening on your answer.
custom_include_list="
/home/*
/usr/src/linux-`uname -r`"

# add files/folders here which are subfolders of a folder listed in $custom_include_list which should NOT
# be backed up. eg.
#custom_exclude_list="/home/foo/mp3 /home/foo/downloads /home/foo/.*"
custom_exclude_list=""

# Only files/folders within the $custom_include_list are checked against these patterns
# custom_exclude_pattern="*.mp3 *.iso"
custom_exclude_pattern=""

# the find_command
find_command="$find /*"

# don't backup anything which matches pattern listed in $default_exclude_pattern
for pattern in $default_exclude_pattern; do
   find_command="$find_command -not -name $pattern"
done

# assemble the find_command
function find_files()
{
   for folder in $default_exclude_list; do
      find_command="$find_command -path $folder -prune -o"
   done

   find_command="$find_command -print"

   for i in $default_include_files; do
      find_command="echo $i; $find_command"
   done   

   for i in $default_include_folders; do
      if [ -d $i ]; then
         find_command="$find $i; $find_command"
      else
         find_command="echo $i; $find_command"
      fi      
   done
}

# check the exclude/include variables for non-existing entries
function verify()
{
   for i in $1; do
      if [ ! -e "`echo "$i" | cut -d'=' -f2 | cut -d'*' -f1`" -a "$i" != "/lost+found" -a "$i" != "$stage4Location" ]; then
         echo "ERROR: `echo "$i" | cut -d'=' -f2` not found! Check your "$2
         exit 0
      fi
   done   
}

# check input parameters
while [ $1 ]; do
   case  $1 in
   "-h" | "--help")
      echo -e $help
      exit 0;;
   "-v" | "--verbose")
      verbose=$1;;
   "-s" | "--split")
      tar_output="--split";;
   "");;
   *)
      echo -e $help
      exit 0;;
   esac
   shift
done

echo ""

# check folder/files listed in $default_exclude_list exist
verify "$default_exclude_list" "\$default_exclude_list"

# check files listed in $default_include_files exist
verify "$default_include_files" "\$default_include_files"

# check folder listed in $default_include_folders exist
verify "$default_include_folders" "\$default_include_folders"

#check folder listed in $custom_include_list exist
verify "$custom_include_list" "\$custom_include_list"

#check folder listed in $custom_exclude_list exist
verify "$custom_exclude_list" "\$custom_exclude_list"
 
# print out the version
 echo -e "\nBackup script $version"
 echo -e "=================="

# how do you want to backup?
echo -e "\nWhat do you want to do? (Use CONTROL-C to abort)\n
Fast (tar.gz):
 (1) Minimal backup
 (2) Interactive backup


Best (tar.bz2):
 (3) Minimal backup
 (4) Interactive backup\n"

while [ "$option" != '1' -a "$option" != '2' -a "$option" != '3' -a "$option" != '4' ]; do
   echo -en "Please enter your option: "
   read option
done

case $option in
[1,3])
   stage4Name=$stage4Location/$stage4prefix-minimal.tar;;

[2,4])
   stage4Name=$stage4Location/$stage4prefix-custom.tar

   for folder in $custom_include_list; do
      echo -en "\nDo you want to backup" `echo "$folder" | cut -d'=' -f2`"? (y/n) "
      read answer
      while [ "$answer" != 'y' -a "$answer" != 'n' ]; do
         echo -en "Do you want to backup" `echo "$folder" | cut -d'=' -f2`"? (y/n) "
         read answer
      done
      if [ "$answer" == 'n' ]; then
         find_command="$find_command -path $folder -prune -o"
      else
         custom_find="$find $folder"
         for i in $custom_exclude_pattern; do
            custom_find="$custom_find -name $i -o"
         done
         for i in $custom_exclude_list; do
            custom_find="$custom_find -path $i -prune -o"
         done
         find_command="$custom_find -print; $find_command"
      fi
   done ;;
esac

# add $custom_include_list to the $default_exclude_list as we assembled
# $custom_find with $custom_include_list already.
default_exclude_list="$default_exclude_list $custom_include_list"

case $option in
[1,2])
   stage4postfix="gz"
   zip="--gzip";;
   
[3,4])
   stage4postfix="bz2"
   zip="--bzip2";;
esac

# mount boot
echo -e "\n* mounting boot"
mount /boot >/dev/null 2>&1

# find the files/folder to backup
find_files
find_command="($find_command)"

# create the final command
if [ "$tar_output" == "--file" ]; then
   tar_command="$find_command | $tar $zip $tarOptions $verbose --file $stage4Name.$stage4postfix --no-recursion -T -"
else
   tar_command="$find_command | $tar $zip $tarOptions $verbose --no-recursion -T - | split $split_options - "$stage4Name.$stage4postfix"_"
fi

if [ "$verbose" ]; then
   echo -ne "\n* creating the stage4 in $stage4Location with the following command:\n\n"$tar_command
fi

# everything is set, are you sure to continue?
echo -ne "\nDo you want to continue? (y/n) "
read answer
while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do
   echo -ne "Do you want to continue? (y/n) "
   read answer
done

if [ "$answer" == 'y' ]; then
   # check whether the file already exists.
   if [ "$tar_output" == "--split" ]; then
      overwrite="`ls "$stage4Name.$stage4postfix"_* 2>&1 | grep -v 'No such file'`"
   else
      overwrite="$stage4Name.$stage4postfix"
   fi
   
   if [ -a "`echo "$overwrite" | grep "$overwrite" -m1`" ]; then 
      echo -en "\nDo you want to overwrite $overwrite? (y/n) "
      read answer
      while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do
         echo -en "Do you want to overwrite $overwrite? (y/n) "
         read answer
      done
      if [ "$answer" == 'n' ]; then
         echo -e "\n* There's nothing to do ... Exiting"
         exit 0;
      fi
   fi
   
   # if necessary, create the stage4Location
   if [ ! -d "$stage4Location" ] ; then
      echo "* creating directory $stage4Location"
      mkdir -p $stage4Location
   fi

   echo -e "\n* Please wait while the stage4 is being created.\n"

   # do the backup.   
   sh -c "$tar_command"
   
   # finished, clean up
   echo -e "\n* stage4 is done"
   echo "* umounting boot"
   umount /boot >/dev/null 2>&1

   # Integrity check
   echo -e "* Checking integrity"
   if [ "$zip" == "--gzip" ]; then
      zip="gzip"
   else
      zip="bzip2"
   fi   
   
   if [ "$tar_output" == "--split" ]; then
      if [ "`cat "$stage4Name.$stage4postfix"_*"" | $zip --test 2>&1`" != "" ]; then
         echo -e "* Integrity check failed. Re-run the script and check your hardware."
         exit -1
      fi
   else
      if [ "`$zip --test  $stage4Name.$stage4postfix 2>&1`" != "" ]; then
         echo -e "* Integrity check failed. Re-run the script and check your hardware."
         exit -1
      fi
   fi
   
   # everything went smoothly"
   echo -e "* Everything went smoothly. You successfully created a stage4."
   
else
   echo -e "\n* There's nothing to do ... Exiting"
fi

# Split the archive into chunks - uncomment the 3 lines if you want to split the stage4
# echo -e "* split $stage4Name.$stage4postfix"
# split $split_options $stage4Name.$stage4postfix "$stage4Name.$stage4postfix"_
# echo "* splitting is done"


7. Download
ftp://blinkeye.ch/gentoo/mkstage4.sh

8. Run it
Code:
wget ftp://blinkeye.ch/gentoo/mkstage4.sh
chmod +x  mkstage4.sh
./mkstage4.sh

this will execute the script (you must be root to succesfully backup all folders).

for available parameters see section "Parameters" in "1. How it works".

9. Restore

Code:
 1. boot off a live-cd and repartition and create filesystems as necessary
 2. eventually reboot, using option: gentoo docache
 3. umount /mnt/cdrom
 4. remove the live-cd and insert the cd with the stage4
 5. mount /dev/cdrom /mnt/cdrom
 6. mount /dev/hdaX /mnt/gentoo
 7. mkdir /mnt/gentoo/boot
 8. mount /dev/hdaX /mnt/gentoo/boot
 9. tar xzvpf /mnt/cdrom/host-stage4-18.04.2005-custom.tar.gz -C /mnt/gentoo/
 or
 9. tar xjvpf /mnt/cdrom/host-stage4-18.04.2005-custom.tar.bz2 -C /mnt/gentoo/
10. mount -t proc none /mnt/gentoo/proc
11. mount -o bind /dev /mnt/gentoo/dev
12. chroot /mnt/gentoo /bin/bash
13. env-update
14. source /etc/profile

if in need adjust necessary files (/etc/fstab, /boot/grub/grub.conf) and/or install grub

15. emerge sync      (rebuild portage tree)
16. exit
17. cd /
18. umount /mnt/cdrom
19. remove backup cd
20. umount /mnt/gentoo/boot
21. umount /mnt/gentoo/dev
22. umount /mnt/gentoo/proc
23  umount /mnt/gentoo
24. Reboot


Re-assemble split-ed stage4
If you split your stage4 either with the command line parameter --split/-s or uncommented the split section within the script you need to re-assemble the split chunks if you intend to restore such a stage4:
Code:
cat stage4.tar.gz_* > stage4.tar.gz

or
Code:
cat stage4.tar.bz2_* > stage4.tar.bz2

substitute stage4.tar.gz/stage4.tar.bz2 with your stage4 name.

10. Changelog
2005-03-23 fixed split command
2005-03-27 added new option for faster backup (gzip)
2005-03-28 as of udev i added /dev/console /dev/null to be backed up - gets rid of those "Unable to open initial console" messages
2005-03-27 added /dev/console, /dev/null to both final_command variables
2005-03-31 fixed issue of included files/folder from excluded parent folder
2005-04-15 fixed typ (zip -> gzip) in command_list
2005-04-20 added new $exclude_pattern variable thanks to a hint by saskatchewan46. not yet checking/verifying valid entries though
2005-04-24 NW RLAS. the stage4 is compressed in-place -> no more temporary additional space is needed. it's faster now.
2005-04-28 Fixed command (bzip2 options now create a bzip2 stage4 again). small change in the code.
2005-04-30 Fixed Restore commands. thanks to Alpo Nestori
2005-05-08 Removed warning about the /boot/boot symlink. no longer necessary
2005-05-09 Added "6. Warnning" section. $default_include_list doesn't recursively back up folders
2005-05-09 Fixed issue with $default_include_list, i.e. added new variable $custom_include_list
2005-05-15 nclude /tmp without its content to preserve folder structure. thanks to Lorijho. Verifying custom_include_list for wrong entries.
2005-06-07 Added integrity check of the stage4. Changed default filename to Year-Month-ay for easier listing.
2005-06-08 Fixed issue about not preserving all permissions.
2005-06-11
* Fixed confusing variable names.
* Removed old debug echo statement.
* Split $default_include_list in $default_include_files and $default_include_folders.
* Added portage's memory (/var/db) to $default_include_folders to prevent any mistakes.
* Added portage's /var/cach/edb to $default_include_folders.
* Included files/folders now take precedence of excluded files/folders.
* Not existing files/folders in any list exits the script as it could be ignored too easily.
* Added ftp link to the script for those having issues copy & pasting it.
* Updated this Howo - explained some variables.

2005-06-13 Added $custom_include_list, $custom_exclude_list, $custom_exclude_pattern to provide the abilty of backing up directories interactively while excluding files/folders deeper in the tree.
2005-06-14
* Added /var/db and /var/cache/edb to the $default_exclude_list to prevent backing it up twice.
* Removed udev warning sections as there seem to be no futher issues.
* Added verbose mode to the script. Default mode is now non-verbose, i.e. only errors are shown. For verbose mode use "mkstage4.sh --verbose" or "mkstage4.sh -v".
* Hardcoded tar and find. Added routine for suggestion path if tar and/or find could not be found.
* Updated Howto. New Section "2. Features". New Section "7. Run it".

2005-06-15 Added section "3. Portability/Prerequesites"
2005-06-21 Added new parameter --split and new section "Parameters" in "1. How it works".
2005-06-30
* Fixed issue with command line parameters and mutual exclusion. --split and --verbose or -s and -v may now be combined in any order.
* Fixed integrity check for splitted files - it's now possible
* Removed /var/cache/edb as this is only cache and portage does not need it to be backed up.
* Removed /var/log/portage per a default as new users get confused about adjusting the script.
* Added used commands to be checked at the beginning.

2005-07-25 Added vmware link to dundas mini howto.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick


Last edited by BlinkEye on Mon Nov 14, 2005 10:59 pm; edited 67 times in total
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1219

PostPosted: Thu Mar 24, 2005 1:13 am    Post subject: Reply with quote

Very usefull!

Thank you!

:wink:
_________________
gentoo user
Back to top
View user's profile Send private message
jdgill0
Veteran
Veteran


Joined: 25 Mar 2003
Posts: 1366
Location: Lexington, Ky -- USA

PostPosted: Thu Mar 24, 2005 3:56 am    Post subject: Reply with quote

You could also backup your MBR (master boot record) with
Code:
dd if=/dev/hdx of=mbr.img bs=512 count=1

where you replace hdx with the dev that contains your MBR for booting, for example hda (not hda1 either).

You could then restore your MBR with
Code:
dd if=mbr.img of=/dev/hda


Although I note that I have not actually ever restored the MBR :)
Back to top
View user's profile Send private message
iverasp
n00b
n00b


Joined: 11 Oct 2004
Posts: 58

PostPosted: Thu Mar 24, 2005 10:49 am    Post subject: Reply with quote

just did a stage4 installation myself, and when rebooting my computer complained about missing /proc folder. so remember to make one with "mkdir -p /proc" before rebooting.
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Thu Mar 24, 2005 11:22 am    Post subject: Reply with quote

iverasp wrote:
just did a stage4 installation myself, and when rebooting my computer complained about missing /proc folder. so remember to make one with "mkdir -p /proc" before rebooting.

you have to do that only if you did exclude wrongly. use
Code:
--exclude=/foldername/*

instead of
Code:
--exclude=/foldername

to keep the directory structure.
note: the -p flag isn't needed anyway, you don't have none existent parent directories of /proc.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Thu Mar 24, 2005 11:26 am    Post subject: Reply with quote

jdgill0 wrote:
You could also backup your MBR (master boot record) with
Code:
dd if=/dev/hdx of=mbr.img bs=512 count=1

where you replace hdx with the dev that contains your MBR for booting, for example hda (not hda1 either).

You could then restore your MBR with
Code:
dd if=mbr.img of=/dev/hda


Although I note that I have not actually ever restored the MBR :)

it's too sensitive.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick


Last edited by BlinkEye on Fri Apr 29, 2005 11:03 pm; edited 1 time in total
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1219

PostPosted: Thu Mar 24, 2005 12:26 pm    Post subject: Reply with quote

jdgill0 wrote:

You could then restore your MBR with
Code:
dd if=mbr.img of=/dev/hda


Although I note that I have not actually ever restored the MBR :)


Well that would overwrite your partition table, too. That's not so good idea if you changed partitions meanwhile or just want to restore it on other hard drive.

As far as I know (so it should be checked), last 2 bytes in the MBR tell BIOS to boot or not to boot from that drive (0xAA55 or something else).
64 bytes before that are partition table: 4 entry with 16 bytes each.
So, boot code (lilo, grub, or whatever) shoud be in the first 446 bytes of MBR.

Maybe we should backup, or at least restore, just boot code and try not to mess partition tables...
Code:
dd if=/dev/hdx of=mbr.img bs=446 count=1


But, anyway, I thing that it should be better to chroot in restored environment, edit fstab & lilo.conf, and run "lilo -v" just to avoid partition location and numbering problems...
_________________
gentoo user


Last edited by tnt on Sun Mar 27, 2005 9:58 pm; edited 1 time in total
Back to top
View user's profile Send private message
wilho
Apprentice
Apprentice


Joined: 22 Jul 2002
Posts: 169

PostPosted: Sun Mar 27, 2005 8:04 pm    Post subject: Reply with quote

You're right according to this: http://www.mobiledyne.com/pub/mbrh.html
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Sun Mar 27, 2005 9:30 pm    Post subject: Reply with quote

thanks for clarifying.

notice: new version of the script - it's now possible to choose between a tar.bzip2 or a tar.gz stage4. i had to unpack a stage4 on a 166Mhz box recently where i noticed it might be desirable not to use bzip2 but gzip.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1219

PostPosted: Sun Mar 27, 2005 10:02 pm    Post subject: Reply with quote

Maybe you should change

Code:
version=v1.2


to

Code:
version=v1.2.1


or something similar...
_________________
gentoo user
Back to top
View user's profile Send private message
tetrahydroc
Tux's lil' helper
Tux's lil' helper


Joined: 22 Aug 2003
Posts: 144

PostPosted: Mon Mar 28, 2005 6:37 am    Post subject: Reply with quote

Been looking for something like this, thanks!
Back to top
View user's profile Send private message
superfes
n00b
n00b


Joined: 03 Apr 2004
Posts: 18
Location: Phoenix, AZ

PostPosted: Tue Mar 29, 2005 12:00 am    Post subject: Missing Quote Reply with quote

I hate to mention this on such a neat script posting and stuff, but you're missing a quote at echo -e "\nBackup script v.1.3

*innocent smile*
_________________
Not today, I've got another pair of pants to deal with.
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Tue Mar 29, 2005 2:34 am    Post subject: Reply with quote

i don't know how this happened, i didn't miss that quote in my script even though i do copy the code from my script. nevertheless i found another minor mistake and hence changed the version to 1.3.1. thanks!
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick


Last edited by BlinkEye on Fri Apr 01, 2005 8:22 pm; edited 1 time in total
Back to top
View user's profile Send private message
grenouille
Tux's lil' helper
Tux's lil' helper


Joined: 12 Jun 2004
Posts: 97

PostPosted: Tue Mar 29, 2005 11:18 am    Post subject: Reply with quote

nice work, thanks :)
Back to top
View user's profile Send private message
syg00
l33t
l33t


Joined: 23 Aug 2004
Posts: 907
Location: Brisbane, AUS

PostPosted: Tue Mar 29, 2005 11:27 am    Post subject: Reply with quote

Had to throw away my / partition today, and this recovered the situation delightfully.
Have some slight issues with the script - hopefully can be dealt with via PM.
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Tue Mar 29, 2005 12:31 pm    Post subject: Reply with quote

glad to hear that. i just noticed that /dev/console and /dev/null don't get backed up this way. doesn't work combined with the exclude parameters. hmm, i'm trying to figure out a solution, but at the moment i don't see one without using the -T parameter (which needs an extra file). anyone has got an idea?
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
tnt
Veteran
Veteran


Joined: 27 Feb 2004
Posts: 1219

PostPosted: Tue Mar 29, 2005 2:07 pm    Post subject: Reply with quote

Why not to make that extra file in /tmp/ right from the script by
Code:
echo "whatever you need" > /tmp/temp.backup.exclude.file

and remove it
Code:
rm -f /tmp/temp.backup.exclude.file

at the and of the script?
_________________
gentoo user
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Thu Mar 31, 2005 9:41 pm    Post subject: Reply with quote

a new version: 1.4.0. i finally took the time to look into the issue that you can't backup files/folders under a folder which is excluded. see info page from tar: it's simply not possible, nomatter how you do it. i added a new variable $default_include_list which purpose's to backup files/folder which are deeper in the tree of an excluded folder (like /var/log/messages or /dev/null). it's somehow slower as i must do it in 3 steps now - and you need temporarly more harddisk memory (the compression of the tar file can't be done in-place anymore, but the final stage4 is of the same size of course). nevertheless the script's now even more easily configurable and you won't get those nasty "Could not open inital root= console" errors with udev anymore. if you want to know the running time use it like:
Code:
time ./mkstage4.sh

_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
mauricev
Apprentice
Apprentice


Joined: 22 Mar 2004
Posts: 193

PostPosted: Fri Apr 01, 2005 1:07 am    Post subject: Stage4 is a success although with a few snags Reply with quote

The backup seems work nicely, but there isn't much information on restore, although I succeeded on a bare metal restore as a test! :D

Anyway, the wiki doesn't seem to mention anything about where to put the stage4 tar file for easy access. I used the steps described in

https://forums.gentoo.org/viewtopic-t-21327-postdays-0-postorder-asc-start-0.html

to build a Live DVD appending the stage4 file I made. This is cool 8)

Also, regarding the wiki, where it says:

Quote:
Partition and mount filesystem


What happened to creating the filesystems? (Here's where I learned that 2005.0 CD for AMD requires ext3 to be mounted as "mount -t ext3...", some bug in the e2fprogs utilities)

For restoration I ran into a number of snags. First, when restoring the partitions, the sequence

Code:
sfdisk -d /dev/(your_disk) > partitions.save


Code:
sfdisk /dev/(your_disk) < partitions.save


works just great if I try onto a second drive in one computer, but when I bring it to another computer, it doesn't work right. I get errors like

Quote:
Partition 1 does not end on cylinder boundary.


The drive where it worked was an Hitachi 400 GB connected via Silicon Image SATA, but the one where it didn't is a WD 74 GB connected via 3Ware SATA). Both systems are Opterons. I'm not sure that could make a difference. In the first case, I was booted on my live filesystem, on the second I was booted from Stage4 DVD.

Interestingly, restoring the MBR after restoring the partitions eliminates partition funkiness, so for some strange reason sfdisk appears to be doing something wrong when it's run from this other computer from my Stage4 DVD (which is based Gentoo 2005.0 CD for AMD64).

However, the MBR didn't completely restore. First, the bootable flag for /dev/sda1 was not set. Second, GRUB just hangs at boot:

Quote:
GRUB


When I tried to fix this with grub-install, it complained

Quote:
/dev/sda does not have a corresponding BIOS drive


I was, however, able to manually run grub and then it just worked.
Back to top
View user's profile Send private message
Greven
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jul 2002
Posts: 138

PostPosted: Fri Apr 01, 2005 2:15 am    Post subject: Reply with quote

You really have made this a great script. How about adding Dar to the script, so if it's finds an older tar, it will dar it and save space.
_________________
veritas vos liberabit...
Linux User Number: 346805
Wine-Wiki
AMD 64 3500+ | MSI "K8T NEO2-FIR" | mushkin Dual Channel DDR 400
Back to top
View user's profile Send private message
mudrii
l33t
l33t


Joined: 26 Jun 2003
Posts: 789
Location: Singapore

PostPosted: Fri Apr 01, 2005 9:36 am    Post subject: Reply with quote

nice script and usefull I will try it
Thanx
_________________
www.gentoo.ro
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Fri Apr 01, 2005 9:58 am    Post subject: Reply with quote

thanks for the feedback.

@mauricev: thanks for the link. i didn't know such a thread existed, have been looking for it for quite some time.
@greven: seems i'll have to look into dar, looks like a great tool.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick
Back to top
View user's profile Send private message
Sade
Guru
Guru


Joined: 22 Mar 2005
Posts: 406
Location: Netherlands - Eindhoven

PostPosted: Fri Apr 01, 2005 12:48 pm    Post subject: Reply with quote

is there a way to easily copy the script from a program such as lynx or links2??

if not, could u put the script in a tar, and post a link to it so i can download it.

when i select copy and paste, i get a lot of spaces in my file, and i don't know how not to select them, or smartly remove the spaces.
_________________
| 1.6Ghz atom N270 | adopt an unanswerd post | a nice way to post a config file |
Back to top
View user's profile Send private message
BlinkEye
Veteran
Veteran


Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums

PostPosted: Fri Apr 01, 2005 1:27 pm    Post subject: Reply with quote

do you know w3m? i like it a lot more than lynx - it has even mouse-scroll, mouse-klick (left and right) support and is coloured!

EDIT: oh my dear - lynx is displaying the gentoo forum really badly. you MUST try w3m, w3m is displaying the page similar to the layout with a graphical browser
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick


Last edited by BlinkEye on Fri Apr 01, 2005 2:19 pm; edited 2 times in total
Back to top
View user's profile Send private message
Sade
Guru
Guru


Joined: 22 Mar 2005
Posts: 406
Location: Netherlands - Eindhoven

PostPosted: Fri Apr 01, 2005 2:11 pm    Post subject: Reply with quote

Thx for the tip on w3m, this is a nice browser. I think u should keep in mind that having a download of the script makes your job harder, because u have 2 update both scripts every time something changes. Copying with w3m is peanuts. So u might want to considder removing the link.

[EDIT]
just wondering: is the ccache also stored in the archive? it would be nice if u had the choise.
_________________
| 1.6Ghz atom N270 | adopt an unanswerd post | a nice way to post a config file |


Last edited by Sade on Fri Apr 01, 2005 2:21 pm; edited 1 time in total
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2, 3 ... 16, 17, 18  Next
Page 1 of 18

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum