HOWTO: Easily make a full system backup (stage4)

Message

someguy · Post by **someguy** » Mon Sep 19, 2005 9:49 pm

wouldnt dd if=/dev/whatever of=/home/whatever.iso bs=512 work ?

iq_zero · Post by **iq_zero** » Tue Sep 20, 2005 8:58 pm

Hello Im having av harddrive that is making a horrible sound so im going to replace it. After looking for easy way of doing this I found this post.
I looks very interesting and I will give it a try.

The link to the script on page 1 is dead please update if the adress is changed i think this would make the wonderful howto complete.

koelle · Post by **koelle** » Wed Sep 21, 2005 7:36 pm

Hi,

i just used the backup script and everything went fine. Now i decided to reinstall WindowsXP. So I first booted the Gentoo 2004.2 Minimal LiveCD and repartitioned my harddisk:

/dev/hda1 WindowsXP as NTFS
/dev/hda2 /boot
/dev/hda3 swap
/dev/hda4 Extended
/dev/hda5 /
/dev/hda6 /usr
/dev/hda7 /opt
/dev/hda8 /var
/dev/hda9 /tmp
/dev/hda10 /home

The whole backup process went fine until I wanted to reinstall grub into my MBR:

grub
root (hd0,1)
setup (hd0,1)

This resulted in the following output

Filesystem type is ext2fs, partition type 0x83

Checking if "/boot/grub/stage1 exists ... yes"
Checking if "/boot/grub/stage2 exists ... yes"
Checking if "/boot/grub/e2fs_stage_1_5 exists ... yes"
Running "embed /boot/grub/e2fs_stage_1_5 (hd0,1) .. . failed (this is not fatal)"

Running "install /boot/grub/stage1 (hd0,1) /boot/grub/stage2 p /boot/grub/menu.lst ... succeeded"

When I try to boot my System grub loads, but only into the grub shell.
How can I get my bootloader working again? I tried all day several times and am desperate right now!

Hope someone can help me.
greetings koelle

koelle · Post by **koelle** » Wed Sep 21, 2005 8:25 pm

Hey,

I finally made it to install grub. I just installed SuSe, then chrooted into my Gentoo Environment and installed grub with:

grub-install --root-directory=/boot /dev/hda

So grub loaded and I could select my Gentoo System. But then suddenly the following message appeared and the booting process stopped:

UDF-fs: no VRS found
kernel panic - not syncing VFS: Unable to mount root fs on unknown-block(3,3)

What does this mean and how can i fix it?

koelle · Post by **koelle** » Thu Sep 22, 2005 8:56 am

Hi,

i solved my problem. I still had root=/dev/hda3 in my grub.conf. It had changed to /dev/hda5. Now it works.

greetings koelle

Hagar · Post by **Hagar** » Sat Sep 24, 2005 10:44 pm

To preserve folder structures you can use the .keep files.

Code: Select all

default_include_files="
/dev/null
/dev/console
/home
`/find /mnt -name .keep`
/proc
/sys
/tmp
/usr/portage
/usr/src
/var/log/emerge.log

In this example the mountpoints will be stored in the backup.
Gentoo provides .keep files in the mountpoints by default (cdrom, floppy) so why not benefit from it.

bingobob · Post by **bingobob** » Sat Oct 01, 2005 2:40 pm

How can i make this automatic (ie non interactive). I'd like to create a cron job to run the script every week.

cheers

bb

DOSBoy · Post by **DOSBoy** » Mon Oct 03, 2005 11:18 pm

You could prepare the user input to create the archive as it is very simple. Make a file with something like:

1
y
n

or whatever. (I don't know exactly how the options proceed). Save it to somefile. Then

sh mkstage4.sh < somefile

should do the trick.

Gotterdammerung · Post by **Gotterdammerung** » Wed Oct 05, 2005 4:25 am

Including me on this thread.

funkoolow · Post by **funkoolow** » Sun Oct 16, 2005 1:26 pm

back here for this simple and direct question: since my gentoo works as webserver with php and mysql, is it relevant that during the backup process it's available on web and so it's possible that db and logs and such can be updated and or modified? will be useful and/or safer to stop apache2-mysql-php?

thanks

Bionut · Post by **Bionut** » Wed Oct 19, 2005 7:20 pm

just wanted to say thanks for the script... I'd hate to loose all my config etc because I'm too lazy to do it all myself.
Cheers

tgh · Post by **tgh** » Fri Oct 21, 2005 3:36 pm

Noob questions.

1) What is a good directory to keep this script in? Where do admins like to keep scripts like this? I'm guessing "/usr/local/sbin" or should it go straight to "/sbin" since it's a critical file needed to restore the system? (Well, not really, you could restore by hand.) A lot of the pages that talk about directory structure lean more towards putting something like this in /usr/local/sbin, if I understand them correctly:

RedHat 5.0 - section 11.1.1.3
Linux Directory Structure (doc.vic.computerbank.org.au)
Linux Directory Structure (www.comptechdoc.org)

(answered my 2nd quesetion)

funkoolow · Post by **funkoolow** » Fri Oct 21, 2005 4:00 pm

funkoolow wrote:back here for this simple and direct question: since my gentoo works as webserver with php and mysql, is it relevant that during the backup process it's available on web and so it's possible that db and logs and such can be updated and or modified? will be useful and/or safer to stop apache2-mysql-php?

thanks

is this question too stupid? cmon, a simple "yes" or "no" would be enough

bigbob73 · Post by **bigbob73** » Fri Oct 21, 2005 4:03 pm

tgh wrote:Noob questions.

1) What is a good directory to keep this script in? Where do admins like to keep scripts like this? I'm guessing "/usr/local/sbin" or should it go straight to "/sbin" since it's a critical file needed to restore the system? (Well, not really, you could restore by hand.) A lot of the pages that talk about directory structure lean more towards putting something like this in /usr/local/sbin, if I understand them correctly:

RedHat 5.0 - section 11.1.1.3
Linux Directory Structure (doc.vic.computerbank.org.au)
Linux Directory Structure (www.comptechdoc.org)

(answered my 2nd quesetion)

I personally keep it in /root/scripts, but you could keep it wherever you feel most comfortable.

Bigbob

iq_zero · Post by **iq_zero** » Mon Nov 07, 2005 7:36 am

funkoolow wrote:
funkoolow wrote:back here for this simple and direct question: since my gentoo works as webserver with php and mysql, is it relevant that during the backup process it's available on web and so it's possible that db and logs and such can be updated and or modified? will be useful and/or safer to stop apache2-mysql-php?

thanks
is this question too stupid? cmon, a simple "yes" or "no" would be enough

I have php and mysql running on my machine. I tryed this backupscript and it worked fine. I didnt stop any servers while creating the backup.
After I had unpacked the stage4 to my disk i rebooted and mysql and some other programs were complaining because i didnt have any log files. I created them with nano and rebooted again just to see that everything was ok.
The backup went great but the disk I was planning to run on crashed 12 hours after the machine was up and running again. IBM Deathstar

I hope something can be useful

funkoolow · Post by **funkoolow** » Mon Nov 07, 2005 12:12 pm

iq_zero wrote:I have php and mysql running on my machine. I tryed this backupscript and it worked fine. I didnt stop any servers while creating the backup.
After I had unpacked the stage4 to my disk i rebooted and mysql and some other programs were complaining because i didnt have any log files. I created them with nano and rebooted again just to see that everything was ok.
The backup went great but the disk I was planning to run on crashed 12 hours after the machine was up and running again. IBM Deathstar
I hope something can be useful

thanks for your reply, man. I asked cause once I had problems too with mysql after restoring a stage4 and I solved them directly remerging mysql. Beyond that, everything gone smooth, so i wondered if this would happen everytime or not (and it seems yes, for now)...

btw, thanks again for your reply (oh, and sorry for your disk, those deskstar are PURE TRASHWARE

)

Kuhrscher · Post by **Kuhrscher** » Sat Nov 12, 2005 5:23 pm

Great script! Thank you!

Lonnie · Post by **Lonnie** » Sun Nov 13, 2005 3:55 am

This may have been answered, but my eyes are starting to cross from reading all the posts, so here goes.
I have a relatively new Gentoo install, only about a couple of weeks old. However, I didn't RTFM nearly as well as I should have. Thus, I have a Very nice partitioning scheme, for a Debian install. Not really very good for a Gentoo install.
What I would like to do is make a complete backup of what I have on here so far.
Totally re-format the hard drive and then restore the system.
I would keep the same partition numbering scheme, but there would be some serious resizing done, and likely make one partition and xfs or reiserfs for use with larger files.
I can burn to DVDs or CDs, and have both available.
So, could I accomplish this by simply commenting out all the "default_exclude_list=" options and changing the destination for the backup?
Here is my df -h output, so a few pointers would also be appreciated, except I will most likely keep all of them.

Filesystem Size Used Avail Use% Mounted on
/dev/hda5 2.8G 178M 2.5G 7% /
udev 501M 212K 501M 1% /dev
/dev/hda6 9.2G 81M 8.7G 1% /home
/dev/hda7 4.6G 3.6G 849M 81% /usr
/dev/hda8 7.4G 228M 6.8G 4% /var
/dev/hda9 4.6G 140M 4.3G 4% /opt
/dev/hda10 8.4G 138M 7.8G 2% /storage
shm 501M 0 501M 0% /dev/shm

Was thinking I could shrink /home down to around 5gig, increase /usr to maybe 8gig, /var and /opt at 1gig each and put all the rest in /storage and make it the xfs or reiserfs to handle any really large files I might run into.

Other than hosing up the initial partitioning scheme, and still having a bit of trouble with playing audio CDs, I am very happy with Gentoo.

Thanks for any and all help.

BlinkEye · Post by **BlinkEye** » Sun Nov 13, 2005 4:06 am

quote from the script itself about $default_exclude_list:

mkstage4.sh wrote:# don't touch it unless you know what you are doing!

you do not know, so leave it. this script works right out of the box for you - it has been made for exactly that purpose. as you'll only resize the partitions you won't even have to change /etc/fstab (well, that wouldn't be a problem, right? you may have to adjust it anyway if you change the filesystem).

for other questions: read at least the first post to get along (while backing up and/or restoring). you eyes won't be starting to cross from that

in case something doesn't work, please remember:

Jan L. A. van de Snepscheut wrote:In theory, there is no difference between theory and practice. But, in practice, there is.

Sheepdogj15 · Post by **Sheepdogj15** » Sun Nov 13, 2005 5:03 am

unless you have security related reasons for being that granular in your partition scheme, i actually would recommend simplifying it a bit. my gentoo workstation is only split between /home and everything else in /. there might be good reason to make /boot a seperate partition (backwards compatibility and such). and sure, if you want make another partition for large files, go ahead.

blinkeye may get annoyed if we talk about this more here, so you may want to just start a new thread elsewhere on the subject.

Lonnie · Post by **Lonnie** » Sun Nov 13, 2005 3:47 pm

Very true, I do not know what my ideas would be doing with your script. So, I will just attempt it. I do have Gentoo installed, so I was able to muddle through setting up fstab at least once.

I did do a trial run with the script and I come up with about a 1gig stage4, and that was with everything that the script would normally accept.

Eyes didn't start crossing until I was into page 6 of 12 of the comments, at around midnight.

For sheepdog, I just like to play with partitions. Had I only made the three partitions suggested, I wouldn't have found this discussion, as I would not have had a need to re-partition my system.

I thank you for your prompt responses, and I assume that what I can't backup with the script is only stuff that will be re-created once I update the system, or in my case, info that I should make copies of before I make the backup attempt.

I really appreciate the hard work that you people do so that non-programmers like myself are able to use such a high quality operating system. Will really be nice once a few others get on the x86_64 bandwagon, like flashplayer.

Thanks again,

Lonnie

BlinkEye · Post by **BlinkEye** » Sun Nov 13, 2005 4:27 pm

hi again. yes, if you haven't fiddled around with serious settings (like changing portage's path and so on) you will not lose any data. there are a lot of things one does not need to backup - because they'll be recreated.

and i do play with partition schemes around a lot myself, and with filesystems too (as i lost a complete reiser4 gentoo installation due to bugs i thought of this backup script).

i completely understand if your eyes are crossing if you read through this hole thread. but this should not be necessary. the thread itself is about problems, questions, suggestions. there's no need to read all the posts. if one does run into "personal" problems you might pm me - this has been done a lot.

i don't get annoyed about posts here at all - i just think it's better for everyone if we talk only about the script and the backup itself to keep the thread consise.

Ejunkie · Post by **Ejunkie** » Mon Nov 14, 2005 12:03 am

hey,

i have a problem with a similar (sorry i hope i wrote is right) script

Code: Select all

EXCLUDE="
--exclude=proc/*
--exclude=sys/*
--exclude=mnt/backup/*
--exclude=mnt/install
--exclude=var/log/*
--exclude=dev/*
--exclude=usr/portage/distfiles/*
"

if [ `whoami` != root ] ; then echo "you must be root, exiting" ; exit ; fi
cd /
tar -cjpvf /mnt/backup/backup.tar.bz2 `echo $EXCLUDE` *

it worked fine and i can get a fully working system with this, but when i want to emerge / compile it fails

and i dont know why, i hope someone can help me out.

Sheepdogj15 · Post by **Sheepdogj15** » Mon Nov 14, 2005 12:53 am

actually hey BlinkEye, i wonder if there's a way to set it up to save your kernal .config file even if you don't save your sources. that if you have to dokernel recomplies later on you don't have to try to remember all the settings you had. i'm sure it's doable, i just don't know enough about bash scripting to try to customize it in

-valheru- · Post by **-valheru-** » Mon Nov 14, 2005 1:46 am

Great script. I had a spare hard drive so figured, why not try it? Script ran perfectly, popped in new drive, followed instructions, seemed smooth till the portgage sync. Looked like it went all the way through however at the end I received this error:

Code: Select all

Updating Portgage cache:
Traceback (most recent callout):
flie "/usr/bin/emerge",line 2705,in)
oldcate=portage.catsplit(cp_list[0])[0]
Index Error:list (forgot this part, can't read my own handwriting)

Anyway, followed the rest oft he instructions and system booted up just like you said. Perhaps b/c of the above error but the restored system seems very sluggish. Programs take a long time to load, etc. Scrolling has intermittent pauses. Any ideas?

Overall, this is a great script and I am sure this has something to do with me and me alone. Thanks.

HOWTO: Easily make a full system backup (stage4)

Total system backup.