System backups

Message

A.S. Pushkin · Post by **A.S. Pushkin** » Wed Mar 31, 2021 3:58 pm

I know this may have been posed previously, but what sort of system
backups, both software and hardware do Gentoo users use..

I think I have a couple of options on hardware. My last build included for my desktop
a hotswap rack,which is a great piece of hardware. I discovered an external device
that is USB3 compatible so I use that for my laptop. So these are options, but I
wonder about using Bluray as well though I am only able to locate 25GB and 50GB
disks.

With these options I wonder what is the best software to use. My desktop runs
Gentoo, but my laptop runs Linux Mint for a convenience reason.

Late last year I think I had a power supply die on me and backing up became more important.
Any suggestions would be appreciated. I should add that I worked as a system assistant admin for a year
back in the mid 1990s. Our serves use DATs for backup, but that is not an option for me.
I occasionally use Clonezilla and Linux Mint offers Timeshift, but I'd like to be consistent
across both boxes without over duplicating data and OS.

Thanks

halcon · Post by **halcon** » Wed Mar 31, 2021 4:11 pm

Hi A.S. Pushkin,

I've been always using rsnapshot. Simple, reliable, configurable.

Post by **pjp** » Wed Mar 31, 2021 4:42 pm

I only worry about data (home, etc, world) and not much else. Especially for binary distros, reinstalling is probably faster than restoring. When I upgrade a system or physical disk, I often make a full copy too.

I've never had a great setup to manage personal backups, so I've always done them adhoc. My problem with data isn't losing it permanently, only having difficulty locating it.

What I'd like to find is a setup that used virtual tapes. I experienced that once, and it worked well. I think it was something like 30-90 days worth of data kept on virtual tape (as well as physical tape). But the virtual tape was obviously easier to access than physical tape.

I've never worried about personal offsite copies. If my house burns down, the data is irrelevant. I have a mobile copy of some stuff for convenience, but not always with me.

Whatever you go with, periodically test that you can restore from it. Even with enterprise solutions, there have been times when the files weren't available to restore.

carcajou · Post by **carcajou** » Wed Mar 31, 2021 8:21 pm

My backup setup is pretty much simple and primitive one.

I use simple shell script with rsync to sync my /home + mkstage4 from Portage to make system snapshot to my secondary HDD (used only for backups). I do it over weekends (when I usually update my system), so backups are performed right before update/upgrade.

There is also an external HDD drive to which I sync only /home once per month.

On work laptop I backup only imporant stuff (= data) to external drive + pCloud.

I also have old MBP lying around, but basically do not have anything imporant on it - so no backups at all.

figueroa · Post by **figueroa** » Thu Apr 01, 2021 3:00 am

I don't really trust backup programs so I roll my own system. I make a stage4 with the following script and exclude file. It works, it's fast, it's frequently tested. (My /home is on a separate partition, and all my partitions are ext4.)

Code: Select all

#!/bin/sh
#stage4.scr
# Also note: https://wiki.archlinux.org/index.php/rsync#Full_system_backup
# (to check for diffs) tar -d -f /mnt/backup0/stage4/stage4.tar.zst -I zstd
# (to restore) tar xpf /mnt/backup0/stage4.tar.zst -I zstd --acls --xattrs-include='*.*' --numeric-owner
mount /mnt/backup0
cd /
date > /mnt/backup0/stage4date.txt
tar cpf /mnt/backup0/stage4.tar.zst -I "zstd -9 -T0" --acls --xattrs --numeric-owner --no-wildcards-match-slash -X /home/USERNAME/bin/exclude.stage4 /
date >> /mnt/backup0/stage4date.txt
cd
umount /mnt/backup0

The exclude file, exclude.stage4:

Code: Select all

dev/*
home/USERNAME/*
home/lost+found
lost+found
media/*
mnt/*/*
proc/*
root/.cache/*
root/.thumbnails/*
root/.dbus/session-bus/*
run/*
sys/*
tmp/*
var/cache/distfiles/*
var/log/portage/*
var/tmp/*

Adjust to match your own layout and preferences. This gives me a 4 G stage4.tar.zst backup. My personal files are backed up separately every night using the same concept. Backup destination is rotated nightly and weekly for my personal files, and weekly + monthly for the system backup.

The above script and exclude is somewhat refined from what I've posted in the forums previously. I have used zstd compression level 19 in the past, but it only saves me about 5% space and takes much more time. Compression level 9 is a good balance.

I would be very pleased to have feedback on the above. I'm always interested in making refinements.

figueroa · Post by **figueroa** » Thu Apr 01, 2021 3:10 am

I'll add that my compressed backups are copied weekly to two USB flash drives (one for personal and one for the system), encrypted with gpg on-the-fly and taken off-site on a four week rotation, however, one of those weekly rotations goes onto my keyring. I do this as part of my Saturday morning routine.

A.S. Pushkin · Post by **A.S. Pushkin** » Thu Apr 01, 2021 3:58 am

Thank you so very much or this feedback. I can not tell you how much I appreciate the information!

Now I have information to put in my book of tech info and a plan/template to start from.

Thank you.

gentoo_ram · Post by **gentoo_ram** » Thu Apr 01, 2021 4:48 pm

I created a backup script that is based on the concepts from Apple's Time Machine implementation. The advantage is that your get a history of your files as they change. That way if you mess up a config file you have a way to get a previous version of it. The disadvantage of this method is the that the data is not compressed. But that's OK, I have a large external hard drive I used for the backups. It's not a big deal that the external drive is slow compared to my internal storage which is mostly SSD at this point.

The key is to use the rsync --link-dest functionality. You can read the man page for more details. But the idea is that files that don't change in your backup directory are hard-linked to the previous copies of the files. So you can make as many "backups" as you want. The files that change are copied to the backup directory. The files in the backup directory that don't change don't take more storage because they are hard-linked to the previous copies.

Note that this script uses a bind-mount of / to pick up the default files in /dev instead of the /dev files generated by the kernel's devtmpfs.

I backup my home directory in a similar way later in my script, but I keep the home directory backup separate. Note that to start this process, you must manually do the first backup and create a "current" symlink to the first backup.

You end up with this:
drwxr-xr-x 25 root root 4096 Jun 25 2020 2021-03-30-03:10:01
drwxr-xr-x 25 root root 4096 Jun 25 2020 2021-03-31-03:10:01
drwxr-xr-x 25 root root 4096 Jun 25 2020 2021-04-01-03:10:01
lrwxrwxrwx 1 root root 19 Apr 1 03:16 current -> 2021-04-01-03:10:01

Occasionally, I go through and delete old backups I don't want anymore.

Here are the key parts of my script:

Code: Select all

rsyncall='rsync -a --info=stats2'
curr_date=`date +%Y-%m-%d-%H:%M:%S`
back_root="/mnt/backup/gateway"
if [ ! -d $back_root/current ] ;
then
        echo 'Current gateway backup directory not found'
else
        echo "Using backup root $back_root"

        root_bind="/mnt/root"

        # Make sure we have our root filesystem bind-point set up
        if [ ! -d $root_bind/bin ] ;
        then
                mount -o bind / $root_bind
        fi

        # Backup most of the filesystem and include /dev from the
        # root bind to get all of the "default" device enties.
        mkdir $back_root/$curr_date
        $rsyncall --exclude-from=/root/gateway-backup-excludes --link-dest=$back_root/current / $back_root/$curr_date
        $rsyncall --link-dest=$back_root/current/dev $root_bind/dev/ $back_root/$curr_date/dev
        rm $back_root/current
        ln -s $curr_date $back_root/current
fi

gateway-backup-excludes:
root/.ccache/*
root/.system/*
sys/*
proc/*
dev/*
mnt/*
run/*
home/*
boot/*
tmp/*
var/tmp/portage/*
usr/portage/*
var/cache/squid/*
var/lib/ntop

figueroa · Post by **figueroa** » Fri Apr 02, 2021 3:33 am

gentoo_ram wrote:I created a backup script that is based on the concepts from Apple's Time Machine implementation. ...

Which is what the Timeshift program does. Very respectable. Having it in your own script(s) gives you much more control.

BTW, if you exclude /mnt/* none of the mounpoints are included in the backup. That's why I use /mnt/*/* which backs up my mountpoints but excludes anything that happens to be currently mounted upon them.

Website · Post by **Zucca** » Fri Apr 02, 2021 8:22 am

This is how I roll currently:

Every two hours rsync a list of files and directories to /var/backup/current
Once a day make a btrfs read-only snapshot of /var/backup/current to /var/backup/<YYYY-MM-DD>
Occasionally insert a hard drive to a doc, which starts an automatic external backup process.

szatox · Post by **szatox** » Fri Apr 02, 2021 10:59 am

I discovered an external device
that is USB3 compatible so I use that for my laptop. So these are options, but I
wonder about using Bluray as well though I am only able to locate 25GB and 50GB
disks.

Well.. Jukeboxes do exist, so you could circumvent the size limitation by using multiple volumes per backup, but quite frankly I'd stick with hard drives. For many reasons, including ease of access, versatility, durability.... Throw some deduplication into the mix and it will be cheaper too. (You can't realistically do that with a jukebox).
Pretty much the only advantage removable volumes have over HDD is that it won't get damaged on rare occasions when a lightning blows up the drive.

My backup solution is a cron job that creates lvm snapshots of the important filesystems (root and /home; portage can be left behind), and then runs rsync with a reference (creates a hardlink to the previous copy if the file hasn't changed, instead of copying it again).

sdauth · Post by **sdauth** » Fri Apr 02, 2021 11:05 am

I use borgbackup on all my machines for "/", daily, everything is stored on my custom NAS (in a RAID1 array)
I like to make a full clone (with "dd") of my SSD as well on a external HDD. At least one by month.
As for regular data (docs, pictures, videos etc..), everything from the main desktop is mirrored to my custom NAS with rsync. Manually and when needed, since it rarely changes.

Tony0945 · Post by **Tony0945** » Mon Jun 07, 2021 9:53 pm

figuroa, I ran shellcheck on your script

Code: Select all

 # shellcheck /usr/local/sbin/makestage4

In /usr/local/sbin/makestage4 line 11:
cd
^-- SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

Did you mean: 
cd || exit

For more information:
  https://www.shellcheck.net/wiki/SC2164 -- Use 'cd ... || exit' or 'cd ... |...

Maybe use pushd/popd?

figueroa · Post by **figueroa** » Mon Jun 07, 2021 11:54 pm

Tony0945 wrote:figueroa, I ran shellcheck on your script
...

Are you sure you mean MY script? My script is completely linear and simple. Did you edit it to match your layout? Yes, in my script I meant cd where it includes cd. It uses cd twice. Just cd / to go to the system root to ensure consistent paths regardless of where it was started, and cd return before exiting. Feel free to leave out the 2nd cd. It's neither necessary nor fatal.

Never used shellcheck.

Tony0945 · Post by **Tony0945** » Tue Jun 08, 2021 12:00 am

I never knew about bare cd

Tony0945 · Post by **Tony0945** » Tue Jun 08, 2021 12:10 am

figueroa wrote:
Tony0945 wrote:figueroa, I ran shellcheck on your script
...
Are you sure you mean MY script? My script is completely linear and simple. Did you edit it to match your layout?

Yes, I think I just changed username to tony.

Spanik · Post by **Spanik** » Tue Jun 08, 2021 6:37 am

Several layers in fact. My data is on a raid5 volume. Mail and virtual machines are on separate disks.

Some directories like photos are regulary copied to a USB drive locally. Mostly when some work is done there like after a weekend of scanning negatives. Likewise with sound recordings, email and VM's.

And then from time to time I rsync the whole raid5 to a USB. There are 2 identical USB disks, one at home and one at the second home. When I go there I rsync and swap the drives. This includes all the other stuff as well.

There is no "backup" for hardware but in a pinch I can use the USB copies on the laptop if needed. Pull a copy of the USB and continue.

figueroa · Post by **figueroa** » Wed Jun 09, 2021 4:13 am

Spanik wrote: ...
And then from time to time I rsync the whole raid5 to a USB. There are 2 identical USB disks, one at home and one at the second home. When I go there I rsync and swap the drives. This includes all the other stuff as well.
...

You might consider a rotation of three rather than two backup destination devices. Safety in redundancy.

Also, have you ever tested a full restoration of the raid5 from the USB copy, maybe on a test device? I have in the past discovered flaws in my backups from attempted restorations, which thankfully were on a test basis.

wjb · Post by **wjb** » Wed Jun 09, 2021 10:45 am

With borgbackup, my daily backup takes about 3 minutes. I keep a rolling weeks worth of these, plus a number of the weeklies. It does deduplication/compression, so at the moment I've got 1.3TB of backups (at ~95GB each) taking up a total of just 73GB. There's all the usual stuff to restore/compare archives, but it can mount individual archives which makes it very easy to cherry pick files to be restored.

WareChameleon · Post by **WareChameleon** » Mon Jun 14, 2021 12:46 am

I have a few scripts for KVM Virtual Machines and Mass Deployment of Diskless Terminals.

https://github.com/CapriG/Gentoo-Automation

Jimmy Jazz · Post by **Jimmy Jazz** » Mon Jun 14, 2021 2:14 pm

A.S. Pushkin wrote:I know this may have been posed previously, but what sort of system
backups, both software and hardware do Gentoo users use..

[..]

Thanks

definitly dirvish

a partition with a lot of inodes (most of the files become links), block of 4k and room

dmpogo · Post by **dmpogo** » Mon Jun 14, 2021 6:21 pm

I backup only /etc and /home on my machines. For the system - well, there is a backup on gentoo site

My main requirement for backups - is that it will work completely unattended, preferably at least for couple of years. Anything which needs any my action 'on weekends' or as part of routine is, unfortunately, as good as no backup for me personally. I will forget, skip, postopone, and it will not be there when needed.

So it happens that I run backuppc. Had instance of office machine not being touched (or looked at), backup wise, for 4 years, and then everything is there, historical snapshots as scheduled, recent incrementals.
Actually, I have configured that instance of backuppcnearly 10 years ago, and did not touch since, except taking offline two machines that died since.

At home one computer initiates backup for itself and all family laptops (my gentoo's and my wife Mac's) when they are available. Backuppc catches them well when they are online and not in 'busy' window of the day

I backup on external drives at home and in the office. Also home machines, laptop, and office desktop contain essentially the same stuff in /home, so there is a fair amount of redundancy. My wife occasionally dumps here stuff on here own external USB disk, so there is redundancy as well.

Tony0945 · Post by **Tony0945** » Mon Jun 14, 2021 7:43 pm

should backup world file too

A.S. Pushkin · Post by **A.S. Pushkin** » Sun Mar 13, 2022 8:42 pm

Apologies for asking what may have already been asked, but in recent months I've begun considering the necessity of a system backup.

This is brought on for several reasons.

I run two distributions, Gentoo and Linux Mint.

I'd appreciate suggestions on how to approach this issue and what is the best software to use.

Gentoo is a tower that includes a hot swap rack while Linux Mint is on a laptop. I have an external device
that allows the use of an external hard drive over USB3.

I've considered deja-dup as well as Grsysnc. My hope is to do separate bakups for /home
as well /root excluding /home.

I have accumulated a great technical library thanks to the Gutenberg project, but projects including
CAD work as well as video/audio editing need to be backed up.

Suggestions on how to clean up /home prior commencing with a backup would be appreciated as well.
FSlint manager looks like what possible tool, but I wonder if there others that might be better.

Suggestions and what others already use would be appreciated.

Thanks

alamahant · Post by **alamahant** » Mon Mar 14, 2022 9:09 pm

I have an external device
that allows the use of an external hard drive over USB3.

What do you mean exactly?
Another machine or just an external usb drive?
In either case use rsync or tar.
Plz request more info if you feel so inclined.

Suggestions on how to clean up /home prior commencing with a backup would be appreciated as well.

you only mean duplicates?

System backups

System backups

Re: System backups

System bakups