Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Offsite backup of large mysql database
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
petterg
Guru
Guru


Joined: 25 Mar 2004
Posts: 500
Location: Oslo, Norway

PostPosted: Sun Jan 28, 2018 10:57 am    Post subject: Offsite backup of large mysql database Reply with quote

My favorite backup software for offsite backups of large files has retired: Obnam
Does anyone here have experience alternatives to Obnam in a large file environment?

I notice a lot of obnam users are switching to Restic. As Restic is a fork of Attic, makes me skeptical. Before I started to use Obnam (April 2015), I used Attic. The problem with Attic was that it took A LOT of cpu and network bandwidth to calculate which parts of the large files to transfer. Basically the calculations on a 40Mbit connection took upto 8 hours with cpu load, while transfering the actual data took less than an hour. I believe the dataset I had at that time was about 4TB, with 2-300MB daily changes. Most of the files were 100-600GB in size (vm-ware disk snapshot files).
With Obnam this all was done in a couple of hours, and the daily increase of required storage space was just a fraction of what Attic required.

The thing with Attic was that it worked well the first few months. So if I should test Restic, it would take quite some time to figure if it has the same issues.

My current need is offsite backup of a mysql database that is expected to grow into 2TB shortly.

My requirements for the backup are:
- push backup
- encryption performed on source side
- daily versions for 30days, monthly versions for a year
- deduplication
- source side knows which fragments that needs to be sent to destination in a timely maner
- (it's OK if setup requires root access on both source and destination servers)

Does anyone have experience with such software?
Back to top
View user's profile Send private message
soitgoes0745
n00b
n00b


Joined: 08 May 2017
Posts: 28
Location: Dallas

PostPosted: Sun Jan 28, 2018 9:18 pm    Post subject: Reply with quote

I have been using borgbackup. I backed up ~500gb and keep incremental backups almost daily. My backups are encrypted and deduplicated both on a remote drive and on a separate local drive. Remounting my backups is very easy, and I have a rotation for so many backups per day, per week and per month. I never used Obnam but I would recommend Borg. I believe it is also an Attic fork.

Best Regards
Back to top
View user's profile Send private message
petterg
Guru
Guru


Joined: 25 Mar 2004
Posts: 500
Location: Oslo, Norway

PostPosted: Mon Jan 29, 2018 10:40 am    Post subject: Reply with quote

Thanks
Reading the history of Borgbackup, I realize the guy behind Borg is the guy I ran into discussions of Attic with back in 2015. I left Attic, he changed it. We were having the same issues, so there's a good chance he'd fixed exactly that. I'll give it a test for a while. If/when it fails I'll come back to this topic.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum