| View previous topic :: View next topic |
| Author |
Message |
BlinkEye
Veteran
Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums
|
 Posted: Mon Feb 21, 2005 5:06 pm Post subject: |
 |
|
if you're still going to use it - i just made a few changes/adjustments.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick |
|
| Back to top |
|
 |
j-m
Retired Dev
Joined: 31 Oct 2004
Posts: 975
|
| Posted: Thu Mar 03, 2005 5:37 pm Post subject: OT |
|
|
BlinkEye, may a suggest an "improvement" to your signature?
| Code: |
"grep -v '^#'" /path/to/config/file
|
People will be scared to use the sed command shown there...  |
|
| Back to top |
|
|
BlinkEye
Veteran
Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums
|
| Posted: Thu Mar 03, 2005 10:09 pm Post subject: |
|
|
well, i'm scared of it too, that's why it's there so to never forget it - it's not even mine!
you're suggestion is one i used for quite some time, but it doesn't achieve exactly the same thing. using your grep commands doesn't remove the white spaces or comments like
of course it's a start 
ps: i just noticed that it needs some improvement though 
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick |
|
| Back to top |
|
|
deprave
n00b
Joined: 14 May 2004
Posts: 63
Location: Flint, Michigan
|
| Posted: Fri Mar 04, 2005 12:38 am Post subject: |
|
|
| great post |
|
| Back to top |
|
|
Jerri
Guru
Joined: 03 Apr 2003
Posts: 353
|
| Posted: Sat Mar 05, 2005 4:40 am Post subject: |
|
|
well, here is my addition... pretty much just a rip off of BlinkEyes post. but why not.. linux is all about choice right :)
Mostly superficial changes, the thing i didn't like about the original, was clearly defined variables (paths file names, etc.) other then that, not much different, I cut out stuff i didn't want/need and changed a few things around.
| Code: | #!/bin/bash
# Full system backup script
# Usage:
# Just run the script, make any changes or to directories excluded, etc. to the script itself
#
version=v1.01
# Variable definitions:
# Max archive size / split size
max_file_size=700
split_file_size=695
# Backup Location / file:
backup_location=/mnt/storage/backup/
backup_name=backup-`date +\%d-\%m-\%Y`.tar.bz2
world_file=backup-`date +\%d-\%m-\%Y`.txt
log_file=backup-`date +\%d-\%m-\%Y`.log
# options for the archive
tar_options="--directory / --create --absolute-names --preserve-permissions --totals --bzip2 --ignore-failed-read --verbose --file"
# Excluded directories list (add any extra directories to exclude)
dir_list="
/usr/src
/home/jer/tmp
/home/jer/download
/usr/share/games"
# Default exluded files / directories:
default_exclude="
--exclude=/tmp/*
--exclude=/var/tmp/*
--exclude=/lost+found/*
--exclude=/dev/*
--exclude=/proc/*
--exclude=/mnt/*
--exclude=/sys/*
--exclude=/usr/portage/*
--exclude=$backup_location"
# Program prompt
echo -e "\nBackup script $version"
echo -e "===================\n"
# Compile list of excluded folders
for i in $dir_list; do
exclude_folder="$exclude_folder --exclude=$i"
done
# Display settings:
#echo -e "Tar options:$tar_options\n"
#echo -e "Excluded folders:\n$exclude_folder $default_exclude\n"
# merge variables:
tar_options="$exclude_folder $default_exclude $tar_options"
# Display tar command:
echo -e "Executing:\n"
echo -e tar $tar_options $backup_location$backup_name "\n"
echo -ne "Do you want to continue? (y/n) "
read answer
# Error checking:
while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do
echo "please enter y or n"
read answer
done
if [ "$answer" == 'n' ]; then
echo -e "\nNothig left to do... Exiting.\n"
exit 0
fi
# check if file already exists
if [ -a "$backup_location$backup_name" ]; then
echo -en "\nOverwrite file $backup_name? (y/n) "
read answer
while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do
echo "please enter y or n"
read answer
done
if [ "$answer" == 'n' ]; then
exit 0;
fi
fi
if [ "$answer" == 'y' ]; then
# mount boot
mount /boot >/dev/null 2>&1
# Create archive
time tar $tar_options $backup_location$backup_name / | tee $backup_location$log_file
# copy the current world file to the stage4 location
echo -e "\nCreating backup overview $world_file"
cp /var/lib/portage/world $backup_location$world_file >/dev/null 2>&1
# Split if archive greater then max_file_size
new_file_size=`du -m $backup_location$backup_name | awk '{ print $1 }'`
if [[ $new_file_size -ge $max_file_size ]]; then
echo -e "\nSplitting archive..."
split --suffix-length=1 --bytes="$split_file_size"m --verbose $backup_location$backup_name $backup_location$backup_name
fi
# Unmount /boot
umount /boot
echo -e "\nBackup is complete.\n"
fi
exit 0
|
|
|
| Back to top |
|
|
Jerri
Guru
Joined: 03 Apr 2003
Posts: 353
|
| Posted: Sat Mar 05, 2005 4:56 pm Post subject: |
|
|
[Edit]: changed the script around a bit after testing. I ran into problems when trying to restore a system, without including /dev in the tar ball. so i included that, i also added a section to create an iso image of the server tar ball. I just copied the contents of the livecd, then add the server tar ball, and burn it all on one disk. works pretty well, I'm working on a restore script, and have been contemplating ripping apart the livecd, so all you have to do is insert the disk, and it will automatically reinstall your whole system.. a sort of bare metal system restor... we shall see [/edit]
this time, I tailored the script to run on a remote computer, (in my case a server operating in a DMZ)
it will make a backup of the entire system, then transfer the files to another computer (preferable one that is more secure). I suppose its not really necessary to have periodic, full backups... but then, I have so few services running on my server, that the archive size is around ~150 megs.
The script comes in two parts:
first part is the actual backup script which will be automatically uploaded to your server.
the second part, is the script that runs the whole deal.
If you want to use it, you will have to change the variables appropriately.
Part 1: (I called this file backup.server.script)
| Code: | #!/bin/bash
# Full system backup script
# (Server Side)
# Variable definitions:
# Backup Location / file:
backup_location=/mnt/backup/
backup_name=backup-`date +\%d-\%m-\%Y`.tar.bz2
world_file=backup-`date +\%d-\%m-\%Y`.txt
log_file=backup-`date +\%d-\%m-\%Y`.log
# options for the archive
tar_options="--directory / --create --absolute-names --preserve-permissions --totals --bzip2 --ignore-failed-read --verbose --file"
# Excluded folders list:
list=""
# Default exluded file/folders:
default_exclude="
--exclude=/tmp/*
--exclude=/var/tmp/*
--exclude=/lost+found/*
--exclude=/proc/*
--exclude=/mnt/*
--exclude=/sys/*
--exclude=/usr/portage/*
--exclude=$backup_location"
# Compile list of excluded folders
for folder in $list; do
exclude_folder="$exclude_folder --exclude=$folder"
done
# merge variables:
tar_options="$exclude_folder $default_exclude $tar_options"
# mount boot
mount /boot >/dev/null 2>&1
# Create archive
echo -e "Backing up system...\n"
time tar $tar_options $backup_location$backup_name / > $backup_location$log_file 2>&1
# copy the current world file to the stage4 location
cp /var/lib/portage/world $backup_location$world_file >/dev/null 2>&1
# Unmount /boot
umount /boot
echo -e "Backup complete."
exit 0 |
Part 2: (run.backup.script)
| Code: | #!/bin/bash
#
# Backup utlity
#client side
storage_location=/mnt/storage/backup/server/
script_local_location=/root/scripts/
restore_script01=restore-sys
restore_script02=restore-sys-cont
# Server side
server_ip=192.168.3.2
script_remote_location=/root/scripts/
backup_location=/mnt/backup/
# File names
script_name=backup.server.script
backup_name=backup-`date +\%d-\%m-\%Y`.tar.bz2
world_file=backup-`date +\%d-\%m-\%Y`.txt
log_file=backup-`date +\%d-\%m-\%Y`.log
temp_iso=/tmp/newgentoo.iso
# upload backup script to remote computer
scp $script_local_location$script_name root@$server_ip:$script_remote_location$script_name
# run backup script on remote computer
ssh -l root $server_ip sh $script_remote_location$script_name
# retreive archives
scp root@$server_ip:$backup_location$backup_name $storage_location$backup_name
scp root@$server_ip:$backup_location$world_file $storage_location$world_file
scp root@$server_ip:$backup_location$log_file $storage_location$log_file
# remove backup script from remote computer
ssh -l root $server_ip rm -f "$backup_location"*
# remove backup archives from remote computer
ssh -l root $server_ip rm -f "$script_remote_location$script_name"
#
# Burn archive
#
# Blank CD:
cdrecord dev=ATAPI:0,0,0 blank=fast gracetime=2 > /dev/null
# Burn archive to livecd:
cp -f $script_local_location$restore_script01 $storage_location
cp -f $script_local_location$restore_script02 $storage_location
cd $storage_location
cd ..
echo -e "Creating ISO image...\n"
mkisofs -quiet -no-emul-boot -boot-load-size 4 -boot-info-table -r -b isolinux/isolinux.bin -c isolinux/boot.catalog -o $temp_iso . > /dev/null
echo -e "Burning image to cd...\n"
cdrecord fs=8m -dao -data dev=ATAPI:0,0,0 speed=10 driveropts=burnfree -overburn gracetime=2 $temp_iso > /dev/null
rm -f $temp_iso
echo -e "\nBack up complete.\n"
exit 0 |
There is a catch to all this, you have to set up public key authentication, so that you can loggin to your server without supplying a password. for more information about this have a look at these articles:
gentoo keychain guide
OpenSSH key management
here
Last edited by Jerri on Sun Mar 06, 2005 7:55 pm; edited 1 time in total |
|
| Back to top |
|
|
BlinkEye
Veteran
Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums
|
| Posted: Sun Mar 06, 2005 12:29 pm Post subject: |
|
|
it was about time i rewrote the script. i did a lot of changes. as usual it can be found here
i liked the version print upon calling the script - thanks Jerry!
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick |
|
| Back to top |
|
|
Jerri
Guru
Joined: 03 Apr 2003
Posts: 353
|
| Posted: Sun Mar 06, 2005 8:00 pm Post subject: |
|
|
| Quote: | | the record was 7CD's for a full installation |
dear me, thats brutal... |
|
| Back to top |
|
|
BlinkEye
Veteran
Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums
|
| Posted: Sun Mar 06, 2005 8:20 pm Post subject: |
|
|
| Jerri wrote: | | Quote: | | the record was 7CD's for a full installation |
dear me, thats brutal... |
i dare not to say it: it didn't work afterwards. the big joke was: 7CD's, but NOT bootable. i first had to setup windows ...
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick |
|
| Back to top |
|
|
thomash
n00b
Joined: 25 Oct 2003
Posts: 14
|
| Posted: Mon Mar 21, 2005 1:18 am Post subject: |
|
|
Hi. I successfully created a stage4 with your script blinkeye.
This is exactly what I've been looking for =)
But I run into some problems when trying to boot from an USB hd.
This is what I did:
---
mkdir /mnt/newsys
mount /dev/sda3 /mnt/newsys
mkdir /mnt/newsys/boot
mount /dev/sda1 /mnt/newsys/boot
cd /mnt/newsys
tar xvjpf ../foo-stage4-19.03.2005-minimal.tar.bz2
---
And I modified my /mnt/newsys/boot/grub/grub.conf to look like this:
---
default 0
timeout 30
splashimage=(hd1,0)/boot/grub/splash.xpm.gz
title=development-sources 2.6.8.1
root (hd1,0)
kernel (hd1,0)/boot/kernel-2.6.8.1 root=/dev/sda3
---
But when powering on my system i get "Operation system not found."
Am i missing something? (also tried (hd0,0) too although this is proboably wrong =) |
|
| Back to top |
|
|
BlinkEye
Veteran
Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums
|
| Posted: Mon Mar 21, 2005 12:44 pm Post subject: |
|
|
i must admit that everytime i install a stage4 to my external hd i screw up both system (the one on the usb drive and the one on the laptop). somehow i get them both back, so, i know what you are talking about. for me it's always the bootloader, and if that works i may still have a wrong /etc/fstab. at the moment i'm not sure where your problem lies, but there are several things you need to check out:
1. check again your path - is your untar-ed stage4 really on /mnt/newsys/ ?
* just a note: creating /mnt/newsys/boot should not be necessary
2. what's in your /mnt/newsys/boot directory? sounds like nothing's in there
3. try to install grub again on /dev/sda
4. what exactly happens after booting your system? does grub show up?
5. you may need to append "rootdelay=5" to your kernel line (without the quotes)
6. are you using udev/devfs?
7. what's your /mnt/newsys/etc/fstab look like?
...
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick |
|
| Back to top |
|
|
thomash
n00b
Joined: 25 Oct 2003
Posts: 14
|
| Posted: Mon Mar 21, 2005 1:12 pm Post subject: |
|
|
Hi. the splashscreen doesn't show at all.
The stage4 is really unpacked to /mnt/newsys/ wich is /dev/sda3 and the kernels+grub is in /mnt/newsys/boot wich is /dev/sda1
You say I don't have to create the /mnt/newsys/boot dir, but how can I then specify that /mnt/newsys/boot should be /dev/sda1 before unpacking?
I read something about creating a new initrd though, not sure if this is necessary or not.
I'll check into it atleast.. how do you manage to screw up your current system in your laptop btw?
And I think i'm using devfs hehe. |
|
| Back to top |
|
|
BlinkEye
Veteran
Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums
|
| Posted: Mon Mar 21, 2005 11:58 pm Post subject: |
|
|
| thomash wrote: | | Hi. the splashscreen doesn't show at all. |
well, this means that grub isn't installed (properly).
| thomash wrote: | | The stage4 is really unpacked to /mnt/newsys/ wich is /dev/sda3 and the kernels+grub is in /mnt/newsys/boot wich is /dev/sda1 |
alright
| thomash wrote: | | You say I don't have to create the /mnt/newsys/boot dir, but how can I then specify that /mnt/newsys/boot should be /dev/sda1 before unpacking? |
sure, forget my comment 
| thomash wrote: | | I read something about creating a new initrd though, not sure if this is necessary or not. |
no, this is not at all necessary. if no splashscreen shows, you don't have grub installed (properly). you did install grub explicitly on /dev/sda, did you?
| Code: | mount -t proc none /mnt/newsys/proc
mount --bind /dev /mnt/newsys/dev (might be wrong, check the gentoo doc)
chroot /mnt/newsys/ /bin/bash
env-update
grub
... (installing grub on /dev/sda - see the gentoo doc) |
| thomash wrote: | | I'll check into it atleast.. how do you manage to screw up your current system in your laptop btw? |
well, somehow if i install grub on the external hd it isn't installed on the laptop anymore (don't know why). as i do not have any disk for my laptop it's quite a problem if i can't boot from it anymore, that's why i do have the external usb hd.
| thomash wrote: | | And I think i'm using devfs hehe. |
in fact shouldn't matter at all. just keep in mind my boot parameter if it won't be able to mount root
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick |
|
| Back to top |
|
|
gungholady
Guru
Joined: 19 Oct 2003
Posts: 392
|
| Posted: Tue Mar 22, 2005 8:48 pm Post subject: |
|
|
Using the script created by BlinkEye, the split command does not work correctly. Here is the error output:
| Code: | Total bytes written: 3584931840 (3.4GiB, 503KiB/s)
real 116m3.419s
user 79m6.556s
sys 2m28.013s
* creating stage4 overview /home/mnt/tarbackup/gungholady-stage4-22.03.2005-custom.txt
* stage4 is done
* umounting boot
split: /home/mnt/tarbackup/home/mnt/tarbackup/gungholady-stage4-22.03.2005-custom: No such file or directory |
The /home/mnt/tarbackup/gungholady-stage4-22.03.2005-custom.tar.bz2 is being correctly created. The error is is the split part of the script. I can't figure out why it is looking for the bz2 file in /mnt/home/tarbackup/home/mnt/tarbackup instead of /home/mnt/tarbackup where it is located. |
|
| Back to top |
|
|
BlinkEye
Veteran
Joined: 21 Oct 2003
Posts: 1046
Location: Gentoo Forums
|
| Posted: Wed Mar 23, 2005 9:53 am Post subject: |
|
|
| gungholady wrote: | Using the script created by BlinkEye, the split command does not work correctly. Here is the error output:
| Code: | Total bytes written: 3584931840 (3.4GiB, 503KiB/s)
real 116m3.419s
user 79m6.556s
sys 2m28.013s
* creating stage4 overview /home/mnt/tarbackup/gungholady-stage4-22.03.2005-custom.txt
* stage4 is done
* umounting boot
split: /home/mnt/tarbackup/home/mnt/tarbackup/gungholady-stage4-22.03.2005-custom: No such file or directory |
The /home/mnt/tarbackup/gungholady-stage4-22.03.2005-custom.tar.bz2 is being correctly created. The error is is the split part of the script. I can't figure out why it is looking for the bz2 file in /mnt/home/tarbackup/home/mnt/tarbackup instead of /home/mnt/tarbackup where it is located. |
thanks a lot for your input. i'm sorry for the inconvenience, i forgot to adjust the split command. it should be
| Code: | | split --suffix-length=1 --bytes=670m $stage4Name.tar.bz2 "$stage4Name"tar.bz2_ && echo "* splitting is done" |
the thing is that after the user decided, if he wants to do a minimal or custom backup $stage4Name gets reassigned with a the full path (but without the extension, "tar.bz2" must be added too.
_________________
Easily backup up your system? klick
Get rid of SSH Brute Force Attempts / Script Kiddies klick |
|
| Back to top |
|
|
gungholady
Guru
Joined: 19 Oct 2003
Posts: 392
|
| Posted: Wed Mar 23, 2005 1:24 pm Post subject: |
|
|
| Thanks. It worked correctly this time. I had tried everything I could think of. |
|
| Back to top |
|
|
Maedhros
Bodhisattva
Joined: 14 Apr 2004
Posts: 5511
Location: Durham, UK
|
| Posted: Wed Mar 23, 2005 2:25 pm Post subject: |
|
|
This thread seems now to be a discussion of BlinkEye's script, so please follow up to here: https://forums.gentoo.org/viewtopic-t-312817.html.
_________________
No-one's more important than the earthworm. |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
