Advice on backup software

[hika]

I would like to ask some advise on backup software.
I recently replaced an old samba etc. server with a new one. The old one used rdiff-backup to daily backup the data internally and about weekly to an external disk. It used rdiffweb, so the users could retrieve data.
While rdiff-backup is well suited, I found out that rdiffweb is no longer actively developed and relies on Python 2.5. An alternative, rdiff-backup-web is even older with pre php 5 code.

Can anybody advise on alternatives. My wishes are:
- automated incremental backup, which doesn't need a graphical frontend. There is no X on the server
- a webbased frontend for retrieval of backed-up files
- Databases (ldap, mysql) are internaly backed up separately, which backups will be added to the weekly external backup (or routed to my own server over vpn)
- maybe at a later date I want to expand to remote backups from Windows 7 workstations

Hika

[gcasillo]

Add me to the list of folks looking for a backup solution particularly the automated incremental backup part. Apparently rdiff-backup, which I have used for the last few years, has been masked as it is not being actively maintained upstream.

[hika]

Looking around, I so far have come up with backuppc as the best solution. I haven't yet really looked into it, but will the upcoming weeks.

Hika

[dahoste]

Currently also on the hunt for a replacement to rdiff-backup, having just been informed by emerge that it's masked due to abandonment and that I'm apparently not 'sane' for continuing to use it.

I swear... sometimes the portage maintainers are just snotty jerks for no good reason.

Anyway, I use rdiff-backup six ways to Sunday, on tons of systems, so am very anxious to get something else suitable chosen and get on with life. If/when I settle on something, I'll check back in here and make a note of it.

[beandog]

I think backup-manager does incremental tarball updates.

[1clue]

1. Add a large removable drive, preferably with esata interface so it's fast.
2. Develop a scheme for backup location, such as /mnt/backups/2014-04-10/
3. Make a script that copies all your precious files from your "hot" drive(s) to the backup, such as cp -prd /home /mnt/backups/2014-04-10/home
4. Every so often, attach the backup disk, create a new folder and run your script.
5. Unmount and detach the backup drive
6. Put the backup drive in some other building.
7. When the drive is full, delete the oldest copy, or prune such that you keep one a month rather than one a week, or one a quarter or whatever. Or retire one backup set and start with new disks.
8. Cycle through more than one physical drive, round-robin style.

I've been doing backups for decades. At one point I used magnetic tape, until such time as I actually had to use the tape. I found out then that, as was more common than you might imagine, the backups were not restorable.

I've gone through almost every conceivable solution, from network rsync style stuff to tarballs to CDROM, RAID etc.

rsync has several disadvantages:

1. You only get a copy of what you have right now; it has no history to undo stupid changes that happened two weeks ago but you just found now.
2. If you're going over a network, you have a significant speed problem for large backups, and you still have to move the backup device out of the building.
3. If you're going across the Internet, you have a huge speed problem.

Tarballs, if the tarball is corrupted you lost the entire backup.

Other proprietary file formats like tapes, you lose your device you lost your backups.

CDROM, well I've found that the ones you burn yourself aren't always that reliable, and anymore you have to spend a lot of money to get a blank that will hold enough data to be worthwhile.

RAID is not a backup solution. Period. It's a way to stay running when a drive fails, or a way to speed up your throughput or expand the size of the drive. Or some combination of the above. RAID protects you from drive failure but not fire, robbery, electrical problems or pretty much every other reason you might want a backup.

So my most reliable solution is the one at the top of the post. You use the most reliable technology you have, a tech that is profoundly available from lots of manufacturers. You have more than one backup device, and it's as fast (or almost as fast) as your normal drives, so you can back up everything you want. It's removable so you can take it with you when you go home.

You have a native hard disk, you store in a native format on a native filesystem, one bad spot on your drive doesn't ruin the entire backup. If you have dated backups of just your data, you can reinstall a system with a new drive or a completely different computer, pull your data back and be done with it. If you need something from a month or a year ago, you have it, and you can do a find without decompression or worrying if your backup device is good for it. More importantly, you can look for it at native hard disk speeds instead of some third-rate storage device speed.
[/b]

[1clue]

Sorry for the double-post.

Whatever solution you use, you need to test a restore. If you can't restore it, it's not a backup. It's a waste of time.

Another thing, you need to have a means to externally recognize your backup media. Something more than A B or C.

Good luck and have fun.

Edit: Rather than triple-post I figured I'd edit. The system you put in place needs to be convenient but not completely automatic. You need to be aware of backups, or you forget about them and how your needs change as time goes on. You need it to be convenient enough to be mostly automatic, but the time you REALLY need convenience is when you need to restore data.

When you back up, your system is probably running normally. When you restore, you have a problem. When you have a problem it means you were trying to access data that is either gone or corrupted. When your data is gone or corrupted and you need it, you need an easy and fast way to get the backup and get your system up with the least headache possible, and the least chance of corruption possible.

[gcasillo]

I've tinkered with rsnapshot this week, and it looks suitable for me. This post in another thread helped explain the configuration file.

[John R. Graham]

... I've been doing backups for decades. At one point I used magnetic tape, until such time as I actually had to use the tape. I found out then that, as was more common than you might imagine, the backups were not restorable. ...

What were you using? My experience has been the reverse, except for the egregiously awful Travan type drives that used the floppy controller. In particular and recently, I've had spectacularly good results with DLT.

I love tape. It's the only way to get a really deep backup history economically. Currently, I use app-backup/flexbackup to perform automatic nightly incrementals to tape.

Whatever solution you use, you need to test a restore. If you can't restore it, it's not a backup. It's a waste of time.

Ahh! A kindred spirit.

- John

[lexflex]

1. cp -prd /home /mnt/backups/2014-04-10/home
   [[/b]

That is what I usually do (in one form or another).

However , for me , the important question is if it is possible to also detect changes ( i.e. how long should I keep old backups? ).
So: my question would be: Does any of the proposed tools do that ?

Disk-crashes are relatively easy: you know your data is destroyed, so you restore your last backup before that moment and take your loss ( couple of days, weeks, or months).

I am mostly worried about the possibility files might get corrupted. Is there a way to create a backup (tar) file and check some kind of checksum ?
( not only for the changed files, which is obvious, but I would like to be allerted if an 'old' file suddenly changed).

So, preferably I would like to make a backup, compare it with my last backup, and then be alerted about changes that involve files that where already there in the old backup ( the new files will obviously be 'new' ) .
Any advice on that?


Or, is the only way to keep at least one backup per year or something like that?

Alex.

[1clue]

... I've been doing backups for decades. At one point I used magnetic tape, until such time as I actually had to use the tape. I found out then that, as was more common than you might imagine, the backups were not restorable. ...

What were you using? My experience has been the reverse, except for the egregiously awful Travan type drives that used the floppy controller. In particular and recently, I've had spectacularly good results with DLT.

I love tape. It's the only way to get a really deep backup history economically. Currently, I use app-backup/flexbackup to perform automatic nightly incrementals to tape.

I have no idea what the brand was anymore, I haven't used tape for probably 20 years. It was a DAT drive and I think it was SCSI. That was the most recent, but I've been using tape since back in the 9-track (open reel) days. At least then I knew the data was good because we used to swap tapes between one office and another.

I also used any number of one-of-a-kind removable disks, including the horrible can't-remember-the-name but they were some sort of bernoulli drive, they put out junk and mainstream everybody bought them, looked like a bigger floppy but held more data. It was so bad they didn't answer their customer support numbers and eventually had a class action suit that told them to knock it off.

IMO you take into account the speed of the device, tape can't possibly compete. If you take into account the cost of the drive, tape can't possibly compete. For $100 you get more than a terabyte of sata2 or sata3, whatever you system can handle. You have genuinely re-writable random access device which is absolutely no different than a normal hard drive.

You don't need to take into account only the speed of writing the tape, you have to take into account the speed of extracting the information you want. You don't need any special software, but you can use ANY software, including what you write yourself.

[1clue]

1. cp -prd /home /mnt/backups/2014-04-10/home
   [[/b]

That is what I usually do (in one form or another).

However , for me , the important question is if it is possible to also detect changes ( i.e. how long should I keep old backups? ).
So: my question would be: Does any of the proposed tools do that ?

Disk-crashes are relatively easy: you know your data is destroyed, so you restore your last backup before that moment and take your loss ( couple of days, weeks, or months).

I am mostly worried about the possibility files might get corrupted. Is there a way to create a backup (tar) file and check some kind of checksum ?
( not only for the changed files, which is obvious, but I would like to be allerted if an 'old' file suddenly changed).

So, preferably I would like to make a backup, compare it with my last backup, and then be alerted about changes that involve files that where already there in the old backup ( the new files will obviously be 'new' ) .
Any advice on that?


Or, is the only way to keep at least one backup per year or something like that?

Alex.

First, I decide on a frequency of backups. I don't have a network backup plan. I do host by host, except for workstations which is a shared drive, which gets backed up.

Second, I decide how long I need the information. If it's something like 'forever' then you will basically accumulate drives in a box.

Keep in mind that using only a single storage device is risky. I use 3, and store them in different places. Each device is rotated through different buildings once a week. Anything I find incredibly important I'll thow onto another backup set and store it outside of the rotation. FWIW though you need to have a MANAGEABLE system, not overly convoluted.

You could script something to take md5sums of every file in the tree, write the sum and the name of the file into a single file at the root of the tree. That would be the least obtrusive way I can think for my scheme, although I don't bother with it right now.

You can prune backups off the drive like I mentioned in my first post, or you can retire media to a box and get more media.

I use a diff if I'm curious. diff /path/to/live/file path/to/backup/file and it's done.

[John R. Graham]

... If you take into account the cost of the drive, tape can't possibly compete. For $100 you get more than a terabyte of sata2 or sata3, whatever you system can handle. You have genuinely re-writable random access device which is absolutely no different than a normal hard drive.

Well, I don't know. My last purchase of media was for 9, 320GiB SDLT tapes for $45. That's 2.8TiB of storage for less than $50 (cheating a little bit because there was shipping in addition to that, but you get the idea). Plus, when I drop a tape, it generally still works thereafter.

Moreover, I can reproduce the state of my servers on any day for the last two years with retained quarterly snapshots going back significantly further.

- John

[1clue]

OK John,

So how much does it cost for your office to be down? For me, the last time I checked, there were 80 people, about 14 of which were on salary So guess at $3000/hour for the non-salary people, no idea for the ones with salaries. If the server that goes down is something needed by everyone in the office (which there are several) then every minute of downtime costs the pay you need to keep paying all your employees, PLUS the money they're not making while you monkey with tapes (A LOT more than the employee cost), PLUS the cost of the replacement hardware.

I can reproduce a point at any time for databases, but the rest really doesn't matter much in my case.

It's up to you really, to me the decision to spend a few bucks to get a hard disk with sata2 or sata3 speeds depending on the server, and random access is pretty easy.

[Splink]

cpio ?

[hika]

For some reason I've been thrown out of the maillist. MAybe I misclicked the wrong line in the mail.

Anyhow I have been looking into backuppc and for those who joined my question I will here give my experiences. I hope others will join with their further experiences.

First the pros and cons:
Pro:
- It has a very nice and user-friendly web-interface. As admin you can manage everything, the users can only see the backups you give them rights to and themselves restore what they have lost. They can only change the settings you allow them to.
- It is very efficient at storing the data, making consecutive full backups in a way incremental. Identical files are only stored ones and then hard-linked. So running an immediate second full backup will take no extra space, save for administrative overhead.

Con:
- It is designed to backup workstations to a central server. You can backup from localhost but creating securityrisks in the process. I do advice against using this product to backup the system areas from the server.
- It is for security designed to run as a separate user, not as root. It will have to login to do a backup and you have to provide the credentials. For backup of the local system I think a separate process running as root under chron will be more secure.
- Because of this design apache also needs to run as this user. Starting with apache 2.3 there is a directive 'VHostUser/VHostGroup' to set the user for a vhost to run as. In portage, 2.3 is not present and when I tried 2.4.9-r2 it gave incompatibility issues with perl. So I didn't investigate that. Instead I run this apache server as a separate process on port 8080/8043. I created a modular configuration for this I will add further down. You can also use the apache2-backup script that comes with it. If you use other apache sites, you will have to change the port of either one.
- The application basic design is a bit weird, placing pif and socket files not in /var/run but in the log directory. I adapted the code as shown below for this.

There is an archive function for storing a backup on other media. I haven't yet looked into that.

Minor bug without consequence except filling up your log:

Code: Select all

Missing paratheses around qw(...) statement in
	/usr/lib/BackupPC/Storage/Text.pm	line 301
	/usr/lib/BackupPC/Lib.pm			Line 1420

Moving BackupPC.pid and BackupPC.sock from log directory to /var/run/BackupPC

Code: Select all

Added to /etc/BackupPC/config.pl
	338:$Conf{SockDir}	=	'/var/run/BackupPC'

Added to /usr/bin/BackupPC
	92:		my $SockDir = $bpc->SockDir();
Changed in /usr/bin/BackupPC
	367:	unlink("$SockDir/BackupPC.pid");
	368:	if ( open(PID, ">", "$SockDir/BackupPC.pid") ) {
	371:		chmod(0444, "$SockDir/BackupPC.pid");
	1850:		unlink("$SockDir/BackupPC.pid");
	1889:		my $sockFile = "$SockDir/BackupPC.sock";
	1972:	unlink("$SockDir/BackupPC.pid");

Added to /usr/lib/BackupPC/Lib.pm
	119:			SockDir	=> '/var/run/BackupPC',
	128:			SockDir	=> '/var/run/BackupPC',
	192:sub SockDir
		{
			my($bpc) = @_;
			return $bpc->{SockDir};
		}
Changed in /usr/lib/BackupPC/Lib.pm
	152:	foreach my $dir ( qw(... SockDir) ) {
	697:	my $SockFile = "$bpc->{SockDir}/BackupPC.sock";

Changed in /usr/lib/BackupPC/CGI/Lib.pm
	47:		use vars qw(... $SockDir);
	79:				$Cgi %In ... $SockDir
Added to /usr/lib/BackupPC/CGI/Lib.pm
	101:	$SockDir = $bpc->SockDir();
	111:	$SockDir = $bpc->SockDir();

Added to /usr/lib/BackupPC/Config/Meta.pm
	117:	SockDir		=>	"string";

Setting up the webserver:
As said I created a common apache.conf and use the directives DEFAULT_CONF and BACKUP_CONF to steer to the specific parts. I found the order of directives to be important. If not right, the second server will fail because it sees the processes of the first server. Following configuration files are mainly extracted from the default apache and backuppc config. Anybody with suggestions is welcome.

/etc/apache2/apache.conf

Code: Select all

ServerRoot "/usr/lib64/apache2"
HostnameLookups Off
ServerName 192.168.222.8

LoadModule actions_module modules/mod_actions.so
	.
	.
	.
LoadModule vhost_alias_module modules/mod_vhost_alias.so

Include /etc/apache2/modules.d/*.conf
<IfDefine DEFAULT_CONF>
	User apache
	Group apache
	LockFile /var/run/apache2/apache.lock
	PidFile /var/run/apache2/apache.pid
	ScoreBoardFile /var/run/apache2/apache.scoreboard
</IfDefine>

<IfDefine BACKUP_CONF>
	User backuppc
	Group backuppc
	LockFile /var/run/BackupPC/apache.lock
	PidFile /var/run/BackupPC/apache.pid
	ScoreBoardFile /var/run/BackupPC/apache.scoreboard
</IfDefine>
Include /etc/apache2/vhosts.d/vhosts.conf

/etc/apache2/vhosts.d/vhosts.conf

Code: Select all

<IfDefine DEFAULT_CONF>
	Listen 80
	NameVirtualHost *:80
# The Default vhost
	<IfDefine DEFAULT_VHOST>
		<VirtualHost *:80>
			ServerName localhost
			Include /etc/apache2/vhosts.d/default_vhost.include
			<IfModule mpm_peruser_module>
				ServerEnvironment apache apache
			</IfModule>
		</VirtualHost>
	</IfDefine>
# A vhost without secure connection
	<VirtualHost *:80>
		ServerName sysInfo.home.dkw
		Include /etc/apache2/vhosts.d/sysInfo.include
		<IfModule mpm_peruser_module>
			ServerEnvironment apache apache
		</IfModule>
	</VirtualHost>
# A vhost with secure connection. The ssl.include file will rewrite.
	<VirtualHost *:80>
		ServerName courierAdmin.home.dkw
		<IfDefine SSL>
			<IfModule ssl_module>
				Include /etc/apache2/vhosts.d/ssl.include
			</IfModule>
		</IfDefine>
		Include /etc/apache2/vhosts.d/courierAdmin.include
		<IfModule mpm_peruser_module>
			ServerEnvironment apache apache
		</IfModule>
	</VirtualHost>
# The SSL portion
	<IfDefine SSL>
	<IfModule ssl_module>
		listen 443
		NameVirtualHost *:443
		<IfDefine SSL_DEFAUL_VHOST>
			<VirtualHost *:443>
				ServerName localhost
				Include /etc/apache2/vhosts.d/ssl.include
				Include /etc/apache2/vhosts.d/default_vhost.include
				<IfModule mpm_peruser_module>
					ServerEnvironment apache apache	
				</IfModule>
			</VirtualHost>
		</IfDefine>
		<VirtualHost *:443>
			ServerName courierAdmin.home.dkw
			Include /etc/apache2/vhosts.d/ssl.include
			Include /etc/apache2/vhosts.d/courierAdmin.include
			<IfModule mpm_peruser_module>
				ServerEnvironment apache apache	
			</IfModule>
		</VirtualHost>
	</IfModule>
	</IfDefine>
</IfDefine>

<IfDefine BACKUP_CONF>
	Listen 8080
	NameVirtualHost *:8080

	<IfDefine DEFAULT_VHOST>
		<VirtualHost *:8080>
			ServerName localhost
			Include /etc/apache2/vhosts.d/default_vhost.include
			<IfModule mpm_peruser_module>
				ServerEnvironment apache apache
			</IfModule>
		</VirtualHost>
	</IfDefine>
	<VirtualHost *:8080>
		ServerName Backup.home.dkw
		<IfDefine SSL>
			<IfModule ssl_module>
				Include /etc/apache2/vhosts.d/ssl.include
			</IfModule>
		</IfDefine>
		Include /etc/apache2/vhosts.d/backup.include
		<IfModule mpm_peruser_module>
			ServerEnvironment backuppc backuppc
		</IfModule>
	</VirtualHost>

	<IfDefine SSL>
	<IfModule ssl_module>
		listen 8043
		NameVirtualHost *:8043
		<IfDefine SSL_DEFAUL_VHOST>
			<VirtualHost *:8043>
				ServerName localhost
				Include /etc/apache2/vhosts.d/ssl.include
				Include /etc/apache2/vhosts.d/default_vhost.include
				<IfModule mpm_peruser_module>
					ServerEnvironment apache apache	
				</IfModule>
			</VirtualHost>
		</IfDefine>
		<VirtualHost *:8043>
			ServerName Backup.home.dkw
			Include /etc/apache2/vhosts.d/ssl.include
			Include /etc/apache2/vhosts.d/backup.include
			<IfModule mpm_peruser_module>
				ServerEnvironment packuppc packuppc	
			</IfModule>
		</VirtualHost>
	</IfModule>
	</IfDefine>
</IfDefine>

/etc/apache2/vhosts.d/ssl.include

Code: Select all

	ErrorLog /var/log/apache2/ssl_error_log
	<IfModule log_config_module>
		TransferLog /var/log/apache2/ssl_access_log
		CustomLog /var/log/apache2/ssl_request_log \
	 	"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
	</IfModule>
# this makes an http url jump to https:
	<IfDefine DEFAULT_CONF>
		RewriteEngine On
		RewriteCond %{HTTPS} !=on
		RewriteRule ^/(.*) https://%{SERVER_NAME}/$1 [R,L]
	</IfDefine>
	<IfDefine BACKUP_CONF>
		RewriteEngine On
		RewriteCond %{HTTPS} !=on
		RewriteRule ^/(.*) https://%{SERVER_NAME}:8043/$1 [R,L]
	</IfDefine>

	SSLEngine on
	SSLOptions +StrictRequire
	SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
	SSLCertificateFile /etc/ssl/apache2/server.crt
	SSLCertificateKeyFile /etc/ssl/apache2/server.key
	<FilesMatch "\.(cgi|shtml|phtml|php)$">
		SSLOptions +StdEnvVars
	</FilesMatch>
	<IfModule setenvif_module>
		BrowserMatch ".*MSIE.*" \
		nokeepalive ssl-unclean-shutdown \
		downgrade-1.0 force-response-1.0
	</IfModule>

/etc/apache2/vhosts.d/backup.include

Code: Select all

DocumentRoot "/var/www/Backup.home.dkw/htdocs"
# This makes just the basic url jump to the BackupPC_Admin file
RedirectMatch "^/$" /BackupPC_Admin

<Directory "/var/www/Backup.home.dkw/htdocs">
	Options Indexes FollowSymLinks
	AllowOverride None
	<IfDefine SSL>
		<IfModule ssl_module>
			SSLOptions +StdEnvVars
		</IfModule>
	</IfDefine>
	SetHandler perl-script
	PerlResponseHandler ModPerl::Registry
	PerlOptions +ParseHeaders
	Options +ExecCGI
	Order allow,deny
	Allow from all
# I use LDAP for authentication where possible. My users have the same credentials for basic login, mail and here. 
# You can use any apache authentication as long as in the end it gives a username. Backuppc itself manages which 
# users have access to what. In config.pl with $Conf{CgiAdminUsers} = 'backuppc hika'; the admin users and further 
# what configuration items the other users can manage. In hosts.pl where the backups are defined you give a list 
# with users who can access that backup. Changing logged-in user you do with https://<username>@<hostname>:<port>
	AuthType Basic
	AuthName "Backup Admin"
	AuthBasicProvider ldap
	AuthLDAPURL "ldap://ldap.home.dkw:389/dc=home?uid?sub?(objectClass=*)"
	AuthLDAPGroupAttribute memberUid, member
	AuthLDAPGroupAttributeIsDN Off
# Not needed for the user gives the credentials
#	AuthLDAPBindDN
#	AuthLDAPBindPassword
# Not working for some as yet unknown reason
#	Require ldap-group cn=backuppc,ou=Groups,dc=home
#	Require ldap-group cn=smbadmins,ou=Groups,dc=home
	Require valid-user
</Directory>

<IfModule alias_module>
	Alias /image/ "/var/www/Backup.home.dkw/htdocs/image/"
</IfModule>

<Directory "/var/www/Backup.home.dkw/htdocs/image">
	SetHandler None
	Options Indexes FollowSymLinks
	Order allow,deny
	Allow from all
</Directory>

I will in a later post go into the actual backups.

[platojones]

I'm in the same boat as you folks. Thank you hika for that review of backuppc...based on your experiences, just setting it up, it already looks significantly more complex and fiddly for what I want or need. For now I've copied the rdiff-backup ebuilds to my local overlay, but I know that isn't a permanent solution. rsnapshot may be the way to go...but it's still not rdiff-backup. Hell, maybe I should fork it and start maintaining it. It's hard to walk away from a piece of software that does exactly what you need it to do and does it so well.

[hika]

Hell, maybe I should fork it and start maintaining it. It's hard to walk away from a piece of software that does exactly what you need it to do and does it so well.

You have my vote! especially if you take on rdiffweb with it.
Or talk with https://bitbucket.org/jostillmanns/rdif ... /wiki/Home. He recently build an experimental web frontend.

Hika

[Navar]

@jrg, wow, DLTs have really went down in cost in 15 years. I'm kind of surprised they still exist.

OK John, So how much does it cost for your office to be down?

That's exactly what business interruption insurance coverage is for. Especially in the case of catastrophic loss, execs are going to be too busy doing their job to interfere with letting system administrators work on data recovery.

[1clue]

@jrg, wow, DLTs have really went down in cost in 15 years. I'm kind of surprised they still exist.

OK John, So how much does it cost for your office to be down?

That's exactly what business interruption insurance coverage is for. Especially in the case of catastrophic loss, execs are going to be too busy doing their job to interfere with letting system administrators work on data recovery.

I can't believe you said that. Are you sure you don't want to rethink this from a business perspective? There are so many things wrong with this I don't even know where to start.

[Navar]

OK John, So how much does it cost for your office to be down?

That's exactly what business interruption insurance coverage is for. Especially in the case of catastrophic loss, execs are going to be too busy doing their job to interfere with letting system administrators work on data recovery.

I can't believe you said that. Are you sure you don't want to rethink this from a business perspective? There are so many things wrong with this I don't even know where to start.

I can. And any well run business evaluates its risks as much as possible and hedges appropriately. There are always factors out of your control.

[1clue]

Wow again.

You totally missed what I was wowing about above. I suppose that's my bad, considering how terse my post was.

I'm not an opponent of insurance, it's a good idea if your business is above a certain size, or rather if your risk is above a certain point.

What surprises me is that you would rely on that insurance so you can tolerate a slow, possibly kludgy backup system when something far better exists for not much more money, and needs zero special software.

Insurance aside (not my part of the business) the cost of downtime is always significant, insurance or not. Even with the best possible insurance, you have a loss: They won't pay you the full value of your time and anticipated income, and they also generally don't pay full replacement cost of your equipment.

Most importantly, they don't pay you for lost customers.

From my perspective, the ability to get data back rapidly outweighs anything a tape can give you. Random-access backups with no compression give you the ability to reduce your downtime hugely. Insurance or not, that's a big money saver when you need it.

Insurance might pay 60% of your total loss, but if your backup system reduces that gross loss from $10,000 to $2,000 then you just paid for a lot of backup storage.

If you read far enough back in this thread, I also used tapes of all sorts. I had a need to restore from a tape -- which failed -- and then on testing realized my tapes were essentially worthless.

I strongly support disk-based backups, no compression and a simple directory structure for organization of data. It's faster to write, faster to restore and you can (and should) frequently search the data both in order to ensure that your data is good, and to make sure you know what you're doing when crunch time comes.

[Navar]

Wow again.
You totally missed what I was wowing about above. I suppose that's my bad, considering how terse my post was.

I'm trying to overlook the overt wow euphemisms.

What surprises me is that you would rely on that insurance so you can tolerate a slow, possibly kludgy backup system when something far better exists for not much more money, and needs zero special software.

Budgets, which have been ever decreasing over the last 15 years. Automated after hours backup system wasn't slow, unless you feel full checksum verification with logs is slow. A full restore didn't take that long. A good business retains capital to back downtime operating risks.

Tape use for me, in a professional setting, was between 1996-2000 as a system admin, please keep that in perspective. I have not used it since. During that time, the medium proved reliable. Given that time frame, what is this superior and cost effective technology then that you speak of to use instead?

Insurance aside (not my part of the business) the cost of downtime is always significant, insurance or not. Even with the best possible insurance, you have a loss: They won't pay you the full value of your time and anticipated income, and they also generally don't pay full replacement cost of your equipment.

Companies and policies vary. Proof of claims? The generality is something to the effect of actual loss of business income you sustain due to the necessary suspension of your “operations” during the period of “restoration.”

Most importantly, they don't pay you for lost customers.

Kinda reaching there? If you have serious loss you're hoping just to survive the aftermath to recover. Those without these hedges to support generally are gone, sometimes sinking their owners financially with them.

From my perspective, the ability to get data back rapidly outweighs anything a tape can give you. Random-access backups with no compression give you the ability to reduce your downtime hugely. Insurance or not, that's a big money saver when you need it.

Interesting perspective, from the sake of convenience. You have offline backup setup this way? And what are the clients that have this need?

Business insurance protection was just one piece of the pie. Nowhere did I claim it was a sole reliance. Just like the rest of the overall business plan, such as data backup, which involves budget allocation towards hardware/support services. I was jumping in after your poke at jrg there for using cheap DLTs to simply point out a hole in your argument. What's more important is frequency of data backups on needs/ability and reliability. When the chips fall, who cares if one was even 20 percent faster on full recovery than the other if the data is faulty? If the ongoing business concern is speed of a particular file recovery due to pebkac somewhere in the organization that is going to cause you to lose customers--then there are bigger business problems afoot than data backup.

Insurance might pay 60% of your total loss, but if your backup system reduces that gross loss from $10,000 to $2,000 then you just paid for a lot of backup storage.

Interesting numbers. One would arguably ask why anyone would bother buying insurance, ever.

If you read far enough back in this thread, I also used tapes of all sorts. I had a need to restore from a tape -- which failed -- and then on testing realized my tapes were essentially worthless.

Unless you had no rotation and infrequent verifies, I'm unsure what and why happened in your case. Older hardware, dirty heads, someone waved tapes near a large transformer, etc. I've had similar fault with platters, particularly expensive IBM SCSI drives before they got out of the business. When those died in a sudden manner, there was no attempt available to try to raw forensic recovery read, they were full on bricked at firmware/drive spinup. Less than 6 months duty.

[1clue]

Budgets, which have been ever decreasing over the last 15 years. Automated after hours backup system wasn't slow, unless you feel full checksum verification with logs is slow. A full restore didn't take that long. A good business retains capital to back downtime operating risks.

Tape use for me, in a professional setting, was between 1996-2000 as a system admin, please keep that in perspective. I have not used it since. During that time, the medium proved reliable. Given that time frame, what is this superior and cost effective technology then that you speak of to use instead?

Back then, there really was no option except tape. Fast forward to the topic of this thread though, the OP wants to set up a system of backups right now. There's absolutely no way I would recommend a tape for modern backups. That's why I'm being the way I'm being right now.

And in my experience, the full restore is not the issue. It's a partial restore, to some system that probably didn't hold the data in the first place. A partial restore with a tape takes just as long as a full restore does, and you can't search it very effectively.

Show me a tape system with random access and a MTBF of a hard disk. Considering the cost of the tape drive, I don't see that there's a lot of difference in cost between a tape or a stack of disks with a removable SATA bay. If you do it right (if your situation demands that level of redundancy) you can take the hard disk and make a bootable backup, or an exact copy of the drive. Your system went down? Pop out the bad disk and pop in a backup. 5 minutes and you're up.

Companies and policies vary. Proof of claims? The generality is something to the effect of actual loss of business income you sustain due to the necessary suspension of your “operations” during the period of “restoration.”

Have you ever had an insurance claim, for anything at all? Not many policies promise full replacement value, and those cost significantly more than a partial claim. Again, better to have the insurance and not need it.

Most importantly, they don't pay you for lost customers.

Kinda reaching there? If you have serious loss you're hoping just to survive the aftermath to recover. Those without these hedges to support generally are gone, sometimes sinking their owners financially with them.

In every business I've been in IT, the best sales prospect is your existing customer. No amount of cold calling or advertising has the return of keeping the customer happy and getting them to buy more business. I'm sticking my head out on this one a bit but I can't really think of any industry at all where this isn't true, except a case where the thing you buy is so infrequently bought as to be a one-time deal. Even then, word of mouth plays a big part.

Nonetheless, if you just lost a bunch of your customer's data or shut them down for a half a day messing with a backup, they're less likely to be repeat customers. That counts as 'loss of customers' in my book.

Fortunately for me, I've never had that "hoping just to survive" situation, at least due to a backup failure. While I call the ones I had catastrophic, they weren't apocalyptic.

From my perspective, the ability to get data back rapidly outweighs anything a tape can give you. Random-access backups with no compression give you the ability to reduce your downtime hugely. Insurance or not, that's a big money saver when you need it.

Interesting perspective, from the sake of convenience. You have offline backup setup this way? And what are the clients that have this need?

The clients don't know what my backup scheme is, nor do most of the people in my business.

I have multiple systems with an ejectable SATA bay. You stick in a raw internal drive, it's your backup. Ever since about 5 years ago when I first noticed these bays, I've been enamored. They cost generally less than USD $15, but they can be hard to find. I have a printed plan as to which drive to use in sequence, several copies of the printout, at least one per site where drives are stored. The same arrangement you would use for tapes, a backup without off-site storage is useless in the event of a fire or any list of other things, as you said yourself.

The only difference is that the MTBF is high enough I reuse the drives more than you would with a tape. But I keep track of which drive is which, and what date they were purchased. I also have 4 or more drives in the rotation for any system. If a production non-backup system starts getting flaky I'll stop rotating the drives purchased before or near that date for backups. I figure the backup use case is less intensive than the production case, and there's a bunch of drives being rotated through anyway, so I call that an acceptable risk.

Business insurance protection was just one piece of the pie. Nowhere did I claim it was a sole reliance. Just like the rest of the overall business plan, such as data backup, which involves budget allocation towards hardware/support services. I was jumping in after your poke at jrg there for using cheap DLTs to simply point out a hole in your argument. What's more important is frequency of data backups on needs/ability and reliability. When the chips fall, who cares if one was even 20 percent faster on full recovery than the other if the data is faulty? If the ongoing business concern is speed of a particular file recovery due to pebkac somewhere in the organization that is going to cause you to lose customers--then there are bigger business problems afoot than data backup.

We don't really have anything to argue about here. Frequency of backups certainly counts, and IMO it's also important that it's not completely automatic. It needs to be in somebody's face enough that they're aware of the changing needs of the business with respect to those backups. Otherwise you wind up spending money on 'automatic backups' of data that is obsolete and hasn't changed for years, when some new project is not backed up at all.

Again, I'm saying a hard disk is more reliable than a tape. I'm also saying that it's more flexible both in terms of backing data up as well as retrieval (I'll stop bringing up random access for at least 5 minutes).

Moreover I'm going to say that a raw hard disk is more ubiquitously supported than a tape drive. You need no special software to run it, no special software to extract data or write it.

Pebkac: No matter how much you like to avoid it or insist it doesn't exist, it does. Somewhere, somebody is going to do it. Best be prepared for it.

Budget: A tape drive today seems to go for around USD $2k for the low end stuff. You need 2 of them at least, because if the drive fails you're in trouble. The bigger the business the more you need, both for failover and for parallelism. You need to be sure they each can read the tapes they make themselves and can also read the tapes the others make, so let's call them identical for sake of argument. And then figure some amount for a stack of tapes.

$4k buys a lot of hard drives. I'm betting by the time those get old enough to worry about, you probably would have needed a new tape drive anyway, and a new stack of tapes.

Unless you had no rotation and infrequent verifies, I'm unsure what and why happened in your case. Older hardware, dirty heads, someone waved tapes near a large transformer, etc. I've had similar fault with platters, particularly expensive IBM SCSI drives before they got out of the business. When those died in a sudden manner, there was no attempt available to try to raw forensic recovery read, they were full on bricked at firmware/drive spinup. Less than 6 months duty.

Woulda coulda shoulda. I spent quite awhile looking for causes, but in the meantime I pretty much realized that I was hosed. Back then I went to CD burners even though they were quite a bit smaller, and only marginally better reliability.

I recently bought 6 WD green drives, and only one of them still works at all. I'm done with those, and also done buying only one brand/model of drive because of that. Fortunately they weren't in the backup pool.

Fortunately I've never had need for forensic recovery. My tape failures came before they were practical, and since then I haven't had a catastrophic failure for anything without having it on a hdd backup.



If it's OK with you, why don't we wind this up? We've chewed all the flavor out of this thread, when all either of us probably wanted to do is offer good advice for somebody who wants a reliable backup.

[hika]

Navar and Iclue, can you two move your fundamental discussion else! My question was and is about good alternatives for rdiff-backup!
Not about fundamental discussions, that in essence come to nothing. Especially within aforementioned context!

Hika