[Howto] Creation of a Stage 5 archive

[d2_racing]

pardon my ignorance, i've been away awhile but what is stage 5?

stage 1 = everything incl. bootstrap
stage 2 = no bootstrap
stage 3 = precompiled binaries
stage 4 = ?
stage 5 = ??

Is it something to do with liveCDs?

Stage 4 is a backup stage from Blinkeye.
Stage 5 is an another methode to backup a existing installation.

I choose the next number to create this script.

[nth10sd]

ok thanks for the info!

[d2_racing]

No problem

[steveL]

my method to backup/clone a system for transfering to another system of the same -march is simple..

Code: Select all

tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --file stage4.tar.bz2 /

then simply scp that tarball to your new system,.. untar it, then simply cd into the /mnt/gentoo/dev directory and run

Code: Select all

MAKEDEV generic

this will create the necessary nodes. after mount proc & dev then finally chroot to it. that's it; you should know the rest.

Thanks for the info; you say we know the rest, but I'm not looking to do a new install, but copy to a new hard drive. I'm guessing that I do this from a live disk then set up grub and I'm good to go?
Is it ok to do this from a running system (I'm thinking of temp files in the home directory.) Also, it seems like this would be the definitive line?:

Code: Select all

tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --exclude=/tmp --exclude=/var/tmp --include=/dev/null --include=/dev/console --file stage4.tar.bz2 /

[d2_racing]

I think

[likewhoa]

my method to backup/clone a system for transfering to another system of the same -march is simple..

Code: Select all

tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --file stage4.tar.bz2 /

then simply scp that tarball to your new system,.. untar it, then simply cd into the /mnt/gentoo/dev directory and run

Code: Select all

MAKEDEV generic

this will create the necessary nodes. after mount proc & dev then finally chroot to it. that's it; you should know the rest.

Thanks for the info; you say we know the rest, but I'm not looking to do a new install, but copy to a new hard drive. I'm guessing that I do this from a live disk then set up grub and I'm good to go?
Is it ok to do this from a running system (I'm thinking of temp files in the home directory.) Also, it seems like this would be the definitive line?:

Code: Select all

tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --exclude=/tmp --exclude=/var/tmp --include=/dev/null --include=/dev/console --file stage4.tar.bz2 /

Code: Select all

rm -rf /usr/portage/distfiles/* && rm -rf /var/tmp/portage/* && rm -rf /var/log/portage/* && tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --exclude=/tmp --file /stage4.tar.bz2 /

is more like it. then to restore boot with livecd then do the following steps after you have mounted the partition to /mnt/gentoo

Code: Select all

cd /mnt/gentoo && tar xvjpf stage4.tar.bz2 && mkdir {dev,proc,tmp,sys} && chmod 1777 tmp && cd dev && MAKEDEV generic && mount -t proc none /mnt/gentoo/proc && mount -o bind /dev /mnt/gentoo/dev 

then just copy /etc/resolv.conf if needed them chroot to it, setup grub etc.. /sys will be regenerated for you, just add it to fstab.

[d2_racing]

I rewritted the howto, so if you have any comments or update that you want, just post

[likewhoa]

thanks for the guide, a stage4/5 is a great way to clone a Gentoo server and rapidly deploy it on a server with the same arch.

[alexdu]

Yes, send an example with the tar or something else plz

Yeah, your are right, tar is not useful for raw disk images

I guess

Code: Select all

cat /dev/hda | bzip2 | ssh user@host "cat > /home/user/hda.backup.010107.bzip"

could be an example?

[dR0PS]

my method to backup/clone a system for transfering to another system of the same -march is simple..

Code: Select all

tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --file stage4.tar.bz2 /

then simply scp that tarball to your new system,.. untar it, then simply cd into the /mnt/gentoo/dev directory and run

Code: Select all

MAKEDEV generic

this will create the necessary nodes. after mount proc & dev then finally chroot to it. that's it; you should know the rest.

Thanks for the info; you say we know the rest, but I'm not looking to do a new install, but copy to a new hard drive. I'm guessing that I do this from a live disk then set up grub and I'm good to go?
Is it ok to do this from a running system (I'm thinking of temp files in the home directory.) Also, it seems like this would be the definitive line?:

Code: Select all

tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --exclude=/tmp --exclude=/var/tmp --include=/dev/null --include=/dev/console --file stage4.tar.bz2 /

Code: Select all

rm -rf /usr/portage/distfiles/* && rm -rf /var/tmp/portage/* && rm -rf /var/log/portage/* && tar cvjp --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/stage4.tar.bz2 --exclude=/tmp --file /stage4.tar.bz2 /

is more like it. then to restore boot with livecd then do the following steps after you have mounted the partition to /mnt/gentoo

Code: Select all

cd /mnt/gentoo && tar xvjpf stage4.tar.bz2 && mkdir {dev,proc,tmp,sys} && chmod 1777 tmp && cd dev && MAKEDEV generic && mount -t proc none /mnt/gentoo/proc && mount -o bind /dev /mnt/gentoo/dev 

then just copy /etc/resolv.conf if needed them chroot to it, setup grub etc.. /sys will be regenerated for you, just add it to fstab.

for me i got some headache while untarring with these tar options

Code: Select all

cd /mnt/gentoo && tar xvjpf stage4.tar.bz2

i had to use

Code: Select all

cd /mnt/gentoo && tar xvjpfP stage4.tar.bz2

Note the capital P
elsewhise all softlinks that point to sth. with '/' at the target would get lost (pointing to nirvana)
.... so i believe thats important !!!!!

pls, correct if i'm wrong on here but at least only these options worked for me.

dR0PS

[d2_racing]

It's the first time that I heard about this P problem...

[d2_racing]

I will check this out tomorrow

[d2_racing]

For my configuration, I don't need to add the P....

What kind of encoding do you use ?

[Dieter@be]

Why didn't you just edit the variables in the stage 4 script?
The values in there are meant to be defaults and users are able to add and remove directories as they wish

[d2_racing]

Because the Stage 4 doesn't backup a complete partition.

I don't want to include each and every directories in the Stage 4 script.

[Dieter@be]

Are you sure about that?

Of course it does not "backup partitions" (or file systems, which is probably what you mean?) but it recursively backups all the directories (mountpoints) you want if you just configure some variables.

$default_include_folders: folders which need to be included in the backup recursively for a minimal working system.

$custom_include_list: directories which are not imperative for a working system but which may be desirable to be also saved by your backup interactively (like /home or /usr/src/)

Unless the *_exclude_{list,pattern} variables match stuff you need, it should back up everything you want, or am I missing something?

http://blinkeye.ch/mediawiki/index.php/ ... _customise

Also, backing up complete file systems makes a backup less convenient imho (less flexibility, fragmented state of file systems, ...) Especially since creating file systems is so easy to do (you could even script it ) edit: hmm after studying your code a bit more you also need to create the new file system. I honestly don't see the use.

[zxy]

Now you will have a Gentoo back on track.

Hi.

I don't understand
if someone needs to make/restore a *nix snapshot why do not use simple low level coping:

Code: Select all

# cp /dev/hda1 ...
; or even
# cp /dev/hda ...
; later mount LiveCD and do back
# cp ... /dev/hda1
; or
# cp ... /dev/hda

you can pipe it any way you want - tar, scp, bzip...

Also, playing with tools like resize_reiserfs you can shrink or expand your partition.
You can also change the physical order of restored partitions - all you need is to fix /etc/fstab after restore. (and may be somewhere else if you use raw hard disk access somewhere - it's unusual case)
If you want to change something in the saved binary image, you can mount it as a loop device: mount -o loop file... mountpoint.... , while it is a single partition and not compressed or splitted.

Simple cp won't preserve atimes for example (and some other things too), and some software uses that. So when you restore the data you'll find yourself in some problems.
I once read a nice document about this on the internet, but (by murphy) I can't find it now. Maybe more luck later...

EDIT: Indeed... here is the link describing (also) the possible problems with different ways of backing up unix systems: http://www.halfgaar.net/backing-up-unix

[d2_racing]

Thanks for the article

[Roman_Gruber]

Hi

The Stage 5 act like Ghost or Acronis True Image.In fact, you can backup a complete partition.With this stage, you can backup a complete HDD and transfer it to a new one without any problems.The technique can be use by administrator, because in case of a crash, they can restore only the partition that failed.

As I have seen Acronis True Image is mentioned.

Does this script really reconstruct everything?

Does this script restore on a new individual harddisk my windo**s, my ntfs-data-partition, my root partition with ext3, my boot partition with ext2?
What does this script with partioning? Assume i want to restore it to a smaller/bigger hdd? Do I have to use gparted before? Is the MBR proper restored? Is grub proper restored?

Thanks, Roman

[d2_racing]

Ok, the Stage 5 is use to backup a *nix parittion.

So everything from ext2,ext3,ReiserFS etc... can do the job.

But, I didn't test for a FAT32 partition.

Also, you cannot bakcup a NTFS partition with the Stage 5 script and the MBR.

But, Acronis True Image can backup a HDD that constain a NTFS,EXT3,EXT2 and also a MBR without any troubles.

[slycordinator]

1) Why do "rm /usr/portage/distfiles/*"? Instead of removing your distfiles, just have that directory excluded.

2) The major reason that you created this script is because stage4 didn't (previously) copy /dev/console for you and currently doesn't copy /dev/tty1 for you. On the other hand, copying /dev/tty1 can be done with stage4 by adding it to the default include list.

[likewhoa]

for a succesfull boot you only need /dev/{console,null} /dev/tty is not mandatory to have, and udev would generate that anyways for you after the init sequence begins.

[slycordinator]

for a succesfull boot you only need /dev/{console,null} /dev/tty is not mandatory to have, and udev would generate that anyways for you after the init sequence begins.

But apparently it is needed for gensplash.

[likewhoa]

agreed.

[slycordinator]

My point, though, was that a new backup method is unnecessary to get what he wanted.