| View previous topic :: View next topic |
| Author |
Message |
causality
Apprentice
Joined: 03 Jun 2006
Posts: 246
|
 Posted: Wed Mar 05, 2025 12:41 am Post subject: Borg Backup Cron Script |
 |
|
Hello, Gentoo folks!
I recently engaged in a little mini project. I wanted to share this with all of you.
Borg-backup is really great. It's downright phenomenal. It's what I use for basic on-site system backups. I can also upload the encrypted, compressed, deduplicated archives to cloud storage and such for secure off-site backups. However, borg lacks one feature I really would like, so I filled that in with a Bash script. I just wanted the ability to say, "make a new backup, then keep the last three backups (regardless of age), then prune the rest and compact the overall archive." As great as borg-backup is, it has no such feature. So I wrote my own.
My intention was to create a backup script that could be added to a weekly cron job, say, every Tuesday at 2 am local time. I wanted a "fire and forget" solution for local on-site backups.
I tried to add error-checking at each important step, with any error being fatal. This is a backup script. My main non-root user is called "light" and the 8TB external backup drive is mounted at /home/light/Backup. I have REDACTED the borg passphrase contained in this root-owned script. The important paths are held in variables that you can customize. Also, I have a massive media collection mounted at /home/light/media that's larger than the backup device, so I exclude it from system backups.
This assumes you've already issued a "borg create" command at the $BORG_REPO directory and wish to maintain it regularly.
Otherwise, here is the script:
| Code: | #!/bin/bash
BORG_REPO="/home/light/Backup/borg_backup"
export BORG_PASSPHRASE="REDACTED"
export BORG_RELOCATED_REPO_ACCESS_IS_OK=no
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no
BORG_OPTS="--compression auto,lzma,5 --stats --progress"
DATE=$(date --iso-8601)-$(hostname)
BORG_VERSION=""
# Get Borg version
if [[ ! "BORG_VERSION="$(borg --version)"" ]]; then
echo "Error obtaining version information. Is Borg installed and available?"
exit
fi
#Logging function
log_event() {
if [[ ! "$(echo "${DATE} - "$1"" >> /var/log/borg_backup.log) 2>/dev/null" ]]; then
echo Error writing to log file /var/log/borg_backup.log
exit
fi
}
BACKUP_LIST=`borg list $BORG_REPO`
if [[ ! $BACKUP_LIST ]]; then
log_event "Error obtaining list of backups, quitting"
exit
fi
BACKUPS=()
LATEST_THREE=""
ALL_BACKUPS="$(borg list $BORG_REPO | cut -d' ' -f1)"
if [[ ! $ALL_BACKUPS ]]; then
log_event "Error obtaining list of all backups, quitting"
exit
fi
ALL_BACKUPS=`(sort <<< $ALL_BACKUPS)`
DELETE_BACKUPS=""LATEST_THREE=""
get_list() {
if [[ ! $BACKUP_LIST ]]; then
log_event "Error obtaining list"
exit
fi
while IFS= read -r line; do
set -- $line
BACKUPS+=("$1")
done <<< "$BACKUP_LIST"
for ((i=${#BACKUPS[@]}-3; i<${#BACKUPS[@]}; i++)); do
LATEST_THREE="${LATEST_THREE} ${BACKUPS[i]}"
done
LATEST_THREE=`(sort <<< $LATEST_THREE)`
for TEMP_BACKUP in $ALL_BACKUPS; do
IN_LIST=0
for MAYBE in $LATEST_THREE; do
if [[ "${TEMP_BACKUP}" = "${MAYBE}" ]]; then
IN_LIST=1
fi
done
if [ $IN_LIST -eq 0 ]; then
DELETE_BACKUPS="${DELETE_BACKUPS} ${TEMP_BACKUP}"
fi
done
}
make_backup() {
# Log Borg version
log_event "Borg version "$BORG_VERSION""
log_event "Starting backup for "$DATE""
TEST_VAR=$(borg create $BORG_OPTS --exclude /dev \
--exclude /home/light/Backup \
--exclude /home/light/media \
--exclude /home/mediaftp/media \
--exclude /proc \
--exclude /sys \
--exclude /tmp \
--exclude /var/tmp/portage \
"$BORG_REPO"/::FullBackup_{now:%Y-%m-%d} /)
if [[ ! $TEST_VAR ]]; then
log_event "Backup creation failed"
exit
else
log_event "Completed backup for "$DATE""
fi
}
delete_backup_list() {
x=""
if [ -z "$DELETE_BACKUPS" ]; then
log_event "Nothing to prune"
else
for x in ${DELETE_BACKUPS}; do
borg delete "${BORG_REPO}"::"${x}" || log_event "Deletion failure for $X"; exit
done
fi
}
compact_backups() {
PRUNE_TASK="borg compact $BORG_REPO"
if [[ ! $PRUNE_TASK ]]; then
log_event "Error compacting, exiting"
exit
fi
}
--exclude /sys \
--exclude /tmp \
--exclude /var/tmp/portage \
"$BORG_REPO"/::FullBackup_{now:%Y-%m-%d} /)
if [[ ! $TEST_VAR ]]; then
log_event "Backup creation failed"
exit
else
log_event "Completed backup for "$DATE""
fi
}
delete_backup_list() {
x=""
if [ -z "$DELETE_BACKUPS" ]; then
log_event "Nothing to prune"
else
for x in ${DELETE_BACKUPS}; do
borg delete "${BORG_REPO}"::"${x}" || log_event "Deletion failure for $X"; exit
done
fi
}
compact_backups() {
PRUNE_TASK="borg compact $BORG_REPO"
if [[ ! $PRUNE_TASK ]]; then
log_event "Error compacting, exiting"
exit
fi
}
make_backup
get_list
delete_backup_list
compact_backups
log_event "Backup process complete" |
|
|
| Back to top |
|
 |
DawgG
l33t
Joined: 17 Sep 2003
Posts: 878
|
| Posted: Thu Mar 13, 2025 10:58 am Post subject: borg prune --keep-last 3 |
|
|
have you tried
| Code: | borg create (...)
borg prune --keep-last 3
borg compact |
i use it with hosts that do automated backups on start/stop and are not switched on regularly, so when eg keep last three weeks would delete data whne the host was not on for four weeks.
_________________
DUMM KLICKT GUT. |
|
| Back to top |
|
|
wjb
l33t
Joined: 10 Jul 2005
Posts: 653
Location: Fife, Scotland
|
| Posted: Fri Mar 14, 2025 1:22 am Post subject: |
|
|
I run daily backups because relatively little changes day-to-day, so they're quick to create (even if kde/plasma has been updated since the last time) and very convenient for undelete's.
Generally keep the numbers down with the following, but for some computers that are only switched on a couple of times a week there will only be two daily's retained.
| Code: | | borg prune --keep-daily=7 --keep-weekly=4 --keep-monthly=6 --glob-archives='{hostname}-*' |
I use '{hostname}-{now}' for the backup names to remind myself which backup I'm looking at, and I usually use 'borg mount' to mount the repository then cd into the required archive when I want to restore or compare old copies.
I only do a compact operation once a month - because I'm doing daily backups I don't really want the overhead of compacting every day. The daily backups take 3 or 4 minutes and a compact can easily double that. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
