How The Backup Process Has Changed [LWN.net]

How The Backup Process Has Changed

Posted Nov 29, 2007 5:54 UTC (Thu) by jamesh (subscriber, #1159)
In reply to: How The Backup Process Has Changed by paulmfoster
Parent article: How The Backup Process Has Changed

Using rsync the way you've described is fine for mirroring your system's current state.  But
it only gives you one state of the filesystem.  How do you recover a file that you
deleted/modified but only noticed after a few backup cycles?

If that is a requirement, then plain rsync probably isn't enough.

(Log in to post comments)

How The Backup Process Has Changed

Posted Nov 29, 2007 6:15 UTC (Thu) by paulmfoster (subscriber, #17313) [Link]

Personally, I'm not very forgiving to users in cases like this. If they 
don't notice the problem before the next backup (tomorrow), it's gone. 
However, if you want to have that kind of functionality, you could simply 
designate different backup sites for each day, for example. This would be 
comparable to doing a separate backup tape for each day of the week, in 
the old days.

FWIW, tar and dump won't provide the functionality you're talking about 
either, out of the box. And *none* of these options will do "incremental" 
backups (only those files changed get backed up, and backups consist of 
only changed files).

How The Backup Process Has Changed

Posted Nov 29, 2007 7:14 UTC (Thu) by chema (subscriber, #32636) [Link]

Both dump and tar can do incremental backup.
You can control the incremental backup in dump with the backup level. 

And in tar case, you can use the '--after-date DATE', '--newer DATE' options which will make
tar to only store files newer than DATE. Moreover, you can combine find with tar to get a
"more selected" backup. 

BTW I'm quite surprised noone has mentioned Bacula or Amanda.

-- 
Chema

How The Backup Process Has Changed

Posted Nov 29, 2007 19:03 UTC (Thu) by vmole (subscriber, #111) [Link]

Well then, let me promote Bacula (http://www.bacula.org/). It's overkill for backing up a single PC to a spare disk, but once you get more than a few clients, having eveything organized in one spot is much nicer than dealing with a bunch of ad-hoc rsync scripts. Bacula supports a wide variety of tapes, tape changers, and disk-based volumes, and DVDs. You can schedule full backups, differential, and incremental. You can run scripts on the clients before and after backups (to e.g. dump databases, stop and start processes, etc.) Supports Windoze, OSX, and most unix-like OSes as clients; while you can build (or download) the server for Windows, it's not the main development platform.

I used it at work, backing up a wide variety of systems, and use it at home. Very satisfied.

dump incremental backups

Posted Dec 1, 2007 9:10 UTC (Sat) by mennucc1 (subscriber, #14730) [Link]

The way 'dump' does incremental backups is superior to what can be achieved by simply using 
  tar '--after-date DATE', '--newer DATE'


Dump indeed stores also the state of directories, so that incremental backups will keep record
of which files were deleted.

How The Backup Process Has Changed

Posted Nov 29, 2007 15:06 UTC (Thu) by paulmfoster (subscriber, #17313) [Link]

Well, apologies to all those who thought I knew what I was talking about. 
Apparently, incremental backups are more widely supported than I thought. 
I stand corrected. BTW, the mikerubel.org link is outstanding.

How The Backup Process Has Changed

Posted Dec 1, 2007 22:19 UTC (Sat) by giraffedata (subscriber, #1954) [Link]

Personally, I'm not very forgiving to users in cases like this. If they don't notice the problem before the next backup (tomorrow), it's gone.

Odd choice of words. I'd say, "I don't provide backup service to users for cases like this."

And I think that's a huge omission. For my personal data, I have lost far more due to corruption of the primary copy than by the primary copy becoming unreadable. Accidental deletion, naive modification of source code, program run amok, dishonest employee, etc. And of those, the majority was not detectable within 24 hours.

So my backup systems have always concentrated on being able to get old copies of files back, at the expense of being able to recover from a broken disk drive easily.

How The Backup Process Has Changed

Posted Dec 2, 2007 5:31 UTC (Sun) by paulmfoster (subscriber, #17313) [Link]

In my case, my "users" are myself and my wife (our company is just us). 
And I'm *not* very forgiving of us. All my data loss has been my own 
stupid goofs. I know right away, and go back to the backup. I'm more 
concerned with having multiple copies of the same backup, in case of disk 
failure or lightning strike.

But if I had a larger company or an employee who had the expertise to hack 
things, I'd take a view more like yours.

How The Backup Process Has Changed

Posted Dec 2, 2007 23:04 UTC (Sun) by giraffedata (subscriber, #1954) [Link]

About half the time when I destroy data with a stupid goof, the destruction has been backed up by the time I discover my error. Something works fine for years, then I get the bright idea to improve it. A week later, I find out the hard way that it wasn't an improvement. I back up every day.

How The Backup Process Has Changed

Posted Nov 29, 2007 6:43 UTC (Thu) by gera (guest, #43819) [Link]


You should see rdiff-backup. It uses librsync, and does act like a "real" backup system -
preserving copies of modified/deleted files.

How The Backup Process Has Changed

Posted Nov 29, 2007 16:44 UTC (Thu) by daniel-crawford (subscriber, #4099) [Link]

In my experience, rdiff-backup is too dependent on Python (version mismatches, odd crashes),
and not robust enough. Plain rsync behaves in a much more stable way across OSes and versions,
and calling it with a script that uses --backup and --backup-dir (like tridge's 52-week shell
script) will give you all the backup window you want.

How The Backup Process Has Changed

Posted Nov 29, 2007 6:51 UTC (Thu) by elama (subscriber, #262) [Link]

I'm using:

 --backup-dir=/backup/deleted/$DIR.$TODAY

Gives me a full backup of all current files and all changed/deleted files in a separate
directory. To me this looks much more convenient than incremental backups.

How The Backup Process Has Changed

Posted Nov 29, 2007 14:25 UTC (Thu) by tcabot (subscriber, #6656) [Link]

http://www.mikerubel.org/computers/rsync_snapshots/ shows how to use rsync to do efficient
incremental backups.

How The Backup Process Has Changed

Posted Nov 29, 2007 18:19 UTC (Thu) by jschrod (subscriber, #1646) [Link]

The answer to your question is dirvish, at http://www.dirvish.org/.