Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
replacement for app-backup/obnam needed
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
DawgG
l33t
l33t


Joined: 17 Sep 2003
Posts: 797

PostPosted: Tue Nov 07, 2017 4:43 pm    Post subject: replacement for app-backup/obnam needed Reply with quote

i've been using app-backup/obnam for some time now and it is very useful for me. since the project will be shut down by the end of this year https://obnam.org/ i am looking for a replacement with similar features:

    deduplication
    shared repo
    pull-backups (so that many different machines can be backed up to one central host that runs the software and maintains the backups and the machines do not need anything but eg running sshd and an ssh-key)
    push-backups (not so important)
    automatic generation versioning
    mountable archive/repo
    compression
    encryption


i also run rsnapshot but obnam is more flexible and more efficient.
so far i have looked at duplicity and borg but they seem (are?) unable to do pull-backups.
as mentioned above it is important that i do not have to install loads of additional software on the backup-clients.

do you have any ideas?
THX for your input!
_________________
DUMM KLICKT GUT.
Back to top
View user's profile Send private message
swimmer
Veteran
Veteran


Joined: 15 Jul 2002
Posts: 1310
Location: Netherlands

PostPosted: Tue Nov 07, 2017 11:02 pm    Post subject: Reply with quote

Shortly before the announcement I switched already to BorgBackup and never looked back ... it fulfils all the (simple) requirements I have on a backup solution. Your mileage might vary though ...

Hope that helps.
swimmer
Back to top
View user's profile Send private message
mike155
l33t
l33t


Joined: 17 Sep 2010
Posts: 667
Location: Frankfurt, Germany

PostPosted: Tue Nov 07, 2017 11:26 pm    Post subject: Reply with quote

Some more alternatives: https://lwn.net/Articles/731044/
Back to top
View user's profile Send private message
Zucca
Veteran
Veteran


Joined: 14 Jun 2007
Posts: 1374
Location: KUUSANKOSKI, Finland

PostPosted: Tue Nov 07, 2017 11:27 pm    Post subject: I backup this way Reply with quote

What I use is rsync+ssh and btrfs snapshots.


  • deduplication
  • shared repo (not sure what you mean with that)
  • pull-backups
  • push-backups
  • automatic generation versioning (time the backup server take n+1 snapshots in X time)
  • mountable archive/repo (well, it's a filesystem)
  • compression
  • encryption (at least not alone like that, you can place the btrfs filesystem on some cryptfs etc)


The biggest problems with this septup is a) if you don't use btrfs and it's a no-go for you (zfs then?) b) there is no client side encryption possible afaik becuase it's the filesystem that's crypted not the actual files.
_________________
..: Zucca :..

Code:
ERROR: '--failure' is not an option. Aborting...
Back to top
View user's profile Send private message
DawgG
l33t
l33t


Joined: 17 Sep 2003
Posts: 797

PostPosted: Wed Nov 08, 2017 12:33 pm    Post subject: Reply with quote

THX for all your input, i will have a look at some stuff (esp. restic, i think).

as said, borg and duplicity cannot do pull-backups (or am i too stupid??). since the boxes to backup range from enterprise SLES-db-servers to openwrt-matchboxes installing lots of stuff on them is a no-go - hence the pull-requirement.

i really like btrfs and use it extensively for private purposes, but some time ago i've had some nasty experiences with it on some important servers so i am still hesitant about it in production use.

rsnapshot is also nice (i use it as my "other" backup solution) but in some cases too tied to its weekday-scheme - if for some reason you have to pull additional unscheduled backups it can completely mess up your daily/weekly/monthly-thing; also, interrupted backups cannot be resumed.

here's a small comparison on a test-system i've been running for some time; /mnt/backup is btrfs-raid-1 w/out deduplication. latest is a subvol of which snapshot can be taken and contains some hosts rsync-backups (only the important files); obnam contains the shared obnam-repo with a couple of more hosts (almost full filesystem copies)

Code:
phobos ~ # du -hs /mnt/backup/latest /mnt/backup/obnam
150G    /mnt/backup/latest
35G     /mnt/backup/obnam

rsync-hosts:
Code:
phobos ~ # ls /mnt/backup/latest/
bibocash-new  klesk  orbb  phobos  squid  squid2

obnam-clients:
Code:
phobos ~ # obnam --config obnam-stuff/obnam.conf-pull-generic clients
fireproxy.stbnet.org
.stbnet.org
klesk.stbnet.org
terratest.stbnet.org
biboclient-neu-4.stbnet.org
bibocups-neu.stbnet.org
squid.stbnet.org
phobos.stbnet.org
grunt.stbnet.org
teratest.stbnet.org
sarge.stbnet.org
bibocash-new.stbnet.org
phrack.stbnet.org
squid2.stbnet.org
phrack:36222.stbnet.org
orbb.stbnet.org

(this also demonstrates one of obnam's bad features: clients cannot be removed from repo so typos are never forgiven)

generations in obnam-repo:
Code:
phobos ~ # obnam --config obnam-stuff/obnam.conf-pull-generic --client-name orbb.stbnet.org generations
2       2017-10-06 15:46:06 +0100 .. 2017-10-06 16:13:58 +0100 (0 files, 0 bytes)  (checkpoint)
144     2017-10-06 16:14:03 +0100 .. 2017-10-06 16:23:46 +0100 (77782 files, 3082128713 bytes)
330     2017-10-09 13:57:12 +0100 .. 2017-10-09 14:00:33 +0100 (70733 files, 2750593500 bytes)
489     2017-10-09 15:20:22 +0100 .. 2017-10-09 15:23:40 +0100 (70733 files, 2750591273 bytes)
646     2017-10-10 10:30:52 +0100 .. 2017-10-10 10:34:09 +0100 (70732 files, 2750590646 bytes)
803     2017-10-11 10:02:45 +0100 .. 2017-10-11 10:06:02 +0100 (70735 files, 2750590687 bytes)
960     2017-10-12 11:14:28 +0100 .. 2017-10-12 11:17:49 +0100 (70732 files, 2750590497 bytes)
1117    2017-10-13 11:54:19 +0100 .. 2017-10-13 11:57:39 +0100 (70732 files, 2750590345 bytes)
1274    2017-10-17 13:30:04 +0100 .. 2017-10-17 13:33:21 +0100 (70741 files, 2750602496 bytes)
1432    2017-10-18 11:00:23 +0100 .. 2017-10-18 11:03:40 +0100 (70738 files, 2750602637 bytes)
1590    2017-10-19 18:06:18 +0100 .. 2017-10-19 18:09:37 +0100 (70740 files, 2750602711 bytes)
1748    2017-10-20 10:43:22 +0100 .. 2017-10-20 10:46:44 +0100 (70738 files, 2750602322 bytes)
1906    2017-10-22 15:15:30 +0100 .. 2017-10-22 15:24:38 +0100 (71211 files, 2916014271 bytes)
2077    2017-10-23 12:23:59 +0100 .. 2017-10-23 12:28:58 +0100 (71457 files, 2938090972 bytes)
2252    2017-10-27 11:38:13 +0100 .. 2017-10-27 11:41:48 +0100 (71455 files, 2938095108 bytes)
2415    2017-11-07 11:03:49 +0100 .. 2017-11-07 11:07:18 +0100 (71456 files, 2938092914 bytes)


i hope to find sth. with similar features to obnam.
_________________
DUMM KLICKT GUT.
Back to top
View user's profile Send private message
Zucca
Veteran
Veteran


Joined: 14 Jun 2007
Posts: 1374
Location: KUUSANKOSKI, Finland

PostPosted: Wed Nov 08, 2017 2:15 pm    Post subject: Reply with quote

DawgG wrote:
i am still hesitant about it in production use.
Yeah. I wouldn't use btrfs in large production environments. And even in personal use you'd need to stick using raid1 or raid10 (like I have).
ZFS is more nature, but isn't as flexible in some cituations.

What I like with (fs) snapshot backupping is that you can recover data just by using some basic linux utils. But it comes with limitations.

I used rdiff-backup before. It's very simple, but only offers incremental backups and over network backupping.
_________________
..: Zucca :..

Code:
ERROR: '--failure' is not an option. Aborting...
Back to top
View user's profile Send private message
paluszak
Apprentice
Apprentice


Joined: 28 Jun 2004
Posts: 219
Location: Warsaw, Poland

PostPosted: Tue Dec 26, 2017 11:57 am    Post subject: Reply with quote

DawgG wrote:
THX for all your input, i will have a look at some stuff (esp. restic, i think).

as said, borg and duplicity cannot do pull-backups (or am i too stupid??). since the boxes to backup range from enterprise SLES-db-servers to openwrt-matchboxes installing lots of stuff on them is a no-go - hence the pull-requirement.
(...)


I faced a similar problem and I decided to go with borg. True that borg cannot do pull-backups but you can do it in at least two different ways:
1) install borg on the box to backup and run it remotely with ssh (which is not a pull-backup, I concur, but it works the same)
2) use sshfs to access the box to backup

Item 2 is what obnam did internally, more or less.

J.
Back to top
View user's profile Send private message
DawgG
l33t
l33t


Joined: 17 Sep 2003
Posts: 797

PostPosted: Mon Feb 05, 2018 8:46 am    Post subject: Reply with quote

THX again for your ideas!
after some consideration and testing i think i will go with plain rsync + btrfs-snapshots. it's nice to be able to just use standard-tools to access your backups. the (perceived?) instability of btrfs i will alleviate by using more disks and making more (redundant) backups.
_________________
DUMM KLICKT GUT.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum