System backups

[A.S. Pushkin]

I know this may have been posed previously, but what sort of system
backups, both software and hardware do Gentoo users use..

I think I have a couple of options on hardware. My last build included for my desktop
a hotswap rack,which is a great piece of hardware. I discovered an external device
that is USB3 compatible so I use that for my laptop. So these are options, but I
wonder about using Bluray as well though I am only able to locate 25GB and 50GB
disks.

With these options I wonder what is the best software to use. My desktop runs
Gentoo, but my laptop runs Linux Mint for a convenience reason.

Late last year I think I had a power supply die on me and backing up became more important.
Any suggestions would be appreciated. I should add that I worked as a system assistant admin for a year
back in the mid 1990s. Our serves use DATs for backup, but that is not an option for me.
I occasionally use Clonezilla and Linux Mint offers Timeshift, but I'd like to be consistent
across both boxes without over duplicating data and OS.

Thanks
_________________
ASPushkin

"In a time of universal deceit - telling the truth is a revolutionary act." -- George Orwell

[halcon]

Hi A.S. Pushkin,

I've been always using rsnapshot. Simple, reliable, configurable.
_________________
A wife asks her husband, a programmer:
- Could you please go shopping for me and buy one carton of milk, and if they have eggs, get 6?
He comes back with 6 cartons of milk.
- Why did you buy 6 cartons of milk?
- They had eggs.

[pjp]

I only worry about data (home, etc, world) and not much else. Especially for binary distros, reinstalling is probably faster than restoring. When I upgrade a system or physical disk, I often make a full copy too.

I've never had a great setup to manage personal backups, so I've always done them adhoc. My problem with data isn't losing it permanently, only having difficulty locating it.

What I'd like to find is a setup that used virtual tapes. I experienced that once, and it worked well. I think it was something like 30-90 days worth of data kept on virtual tape (as well as physical tape). But the virtual tape was obviously easier to access than physical tape.

I've never worried about personal offsite copies. If my house burns down, the data is irrelevant. I have a mobile copy of some stuff for convenience, but not always with me.

Whatever you go with, periodically test that you can restore from it. Even with enterprise solutions, there have been times when the files weren't available to restore.
_________________
Quis separabit? Quo animo?

[kukibl]

My backup setup is pretty much simple and primitive one.

I use simple shell script with rsync to sync my /home + mkstage4 from Portage to make system snapshot to my secondary HDD (used only for backups). I do it over weekends (when I usually update my system), so backups are performed right before update/upgrade.

There is also an external HDD drive to which I sync only /home once per month.

On work laptop I backup only imporant stuff (= data) to external drive + pCloud.

I also have old MBP lying around, but basically do not have anything imporant on it - so no backups at all.

[figueroa]

I don't really trust backup programs so I roll my own system. I make a stage4 with the following script and exclude file. It works, it's fast, it's frequently tested. (My /home is on a separate partition, and all my partitions are ext4.)

[figueroa]

I'll add that my compressed backups are copied weekly to two USB flash drives (one for personal and one for the system), encrypted with gpg on-the-fly and taken off-site on a four week rotation, however, one of those weekly rotations goes onto my keyring. I do this as part of my Saturday morning routine.
_________________
Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi

[A.S. Pushkin]

Thank you so very much or this feedback. I can not tell you how much I appreciate the information!

Now I have information to put in my book of tech info and a plan/template to start from.


Thank you.
_________________
ASPushkin

"In a time of universal deceit - telling the truth is a revolutionary act." -- George Orwell

[gentoo_ram]

I created a backup script that is based on the concepts from Apple's Time Machine implementation. The advantage is that your get a history of your files as they change. That way if you mess up a config file you have a way to get a previous version of it. The disadvantage of this method is the that the data is not compressed. But that's OK, I have a large external hard drive I used for the backups. It's not a big deal that the external drive is slow compared to my internal storage which is mostly SSD at this point.

The key is to use the rsync --link-dest functionality. You can read the man page for more details. But the idea is that files that don't change in your backup directory are hard-linked to the previous copies of the files. So you can make as many "backups" as you want. The files that change are copied to the backup directory. The files in the backup directory that don't change don't take more storage because they are hard-linked to the previous copies.

Note that this script uses a bind-mount of / to pick up the default files in /dev instead of the /dev files generated by the kernel's devtmpfs.

I backup my home directory in a similar way later in my script, but I keep the home directory backup separate. Note that to start this process, you must manually do the first backup and create a "current" symlink to the first backup.

You end up with this:
drwxr-xr-x 25 root root 4096 Jun 25 2020 2021-03-30-03:10:01
drwxr-xr-x 25 root root 4096 Jun 25 2020 2021-03-31-03:10:01
drwxr-xr-x 25 root root 4096 Jun 25 2020 2021-04-01-03:10:01
lrwxrwxrwx 1 root root 19 Apr 1 03:16 current -> 2021-04-01-03:10:01

Occasionally, I go through and delete old backups I don't want anymore.

Here are the key parts of my script:

[figueroa]

[Zucca]

This is how I roll currently:

• Every two hours rsync a list of files and directories to /var/backup/current
• Once a day make a btrfs read-only snapshot of /var/backup/current to /var/backup/<YYYY-MM-DD>
• Occasionally insert a hard drive to a doc, which starts an automatic external backup process.

_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

[szatox]

[sdauth]

I use borgbackup on all my machines for "/", daily, everything is stored on my custom NAS (in a RAID1 array)
I like to make a full clone (with "dd") of my SSD as well on a external HDD. At least one by month.
As for regular data (docs, pictures, videos etc..), everything from the main desktop is mirrored to my custom NAS with rsync. Manually and when needed, since it rarely changes.

[Tony0945]

figuroa, I ran shellcheck on your script

[figueroa]

[Tony0945]

I never knew about bare cd

[Tony0945]

[Spanik]

Several layers in fact. My data is on a raid5 volume. Mail and virtual machines are on separate disks.

Some directories like photos are regulary copied to a USB drive locally. Mostly when some work is done there like after a weekend of scanning negatives. Likewise with sound recordings, email and VM's.

And then from time to time I rsync the whole raid5 to a USB. There are 2 identical USB disks, one at home and one at the second home. When I go there I rsync and swap the drives. This includes all the other stuff as well.

There is no "backup" for hardware but in a pinch I can use the USB copies on the laptop if needed. Pull a copy of the USB and continue.
_________________
Expert in non-working solutions

[figueroa]

[wjb]

With borgbackup, my daily backup takes about 3 minutes. I keep a rolling weeks worth of these, plus a number of the weeklies. It does deduplication/compression, so at the moment I've got 1.3TB of backups (at ~95GB each) taking up a total of just 73GB. There's all the usual stuff to restore/compare archives, but it can mount individual archives which makes it very easy to cherry pick files to be restored.

[WareChameleon]

I have a few scripts for KVM Virtual Machines and Mass Deployment of Diskless Terminals.

https://github.com/CapriG/Gentoo-Automation

[Jimmy Jazz]

[dmpogo]

I backup only /etc and /home on my machines. For the system - well, there is a backup on gentoo site My main requirement for backups - is that it will work completely unattended, preferably at least for couple of years. Anything which needs any my action 'on weekends' or as part of routine is, unfortunately, as good as no backup for me personally. I will forget, skip, postopone, and it will not be there when needed.

So it happens that I run backuppc. Had instance of office machine not being touched (or looked at), backup wise, for 4 years, and then everything is there, historical snapshots as scheduled, recent incrementals.
Actually, I have configured that instance of backuppcnearly 10 years ago, and did not touch since, except taking offline two machines that died since.

At home one computer initiates backup for itself and all family laptops (my gentoo's and my wife Mac's) when they are available. Backuppc catches them well when they are online and not in 'busy' window of the day

I backup on external drives at home and in the office. Also home machines, laptop, and office desktop contain essentially the same stuff in /home, so there is a fair amount of redundancy. My wife occasionally dumps here stuff on here own external USB disk, so there is redundancy as well.

[Tony0945]

should backup world file too

[A.S. Pushkin]

Apologies for asking what may have already been asked, but in recent months I've begun considering the necessity of a system backup.

This is brought on for several reasons.

I run two distributions, Gentoo and Linux Mint.

I'd appreciate suggestions on how to approach this issue and what is the best software to use.

Gentoo is a tower that includes a hot swap rack while Linux Mint is on a laptop. I have an external device
that allows the use of an external hard drive over USB3.

I've considered deja-dup as well as Grsysnc. My hope is to do separate bakups for /home
as well /root excluding /home.

I have accumulated a great technical library thanks to the Gutenberg project, but projects including
CAD work as well as video/audio editing need to be backed up.

Suggestions on how to clean up /home prior commencing with a backup would be appreciated as well.
FSlint manager looks like what possible tool, but I wonder if there others that might be better.


Suggestions and what others already use would be appreciated.

Thanks
_________________
ASPushkin

"In a time of universal deceit - telling the truth is a revolutionary act." -- George Orwell

[alamahant]