How The Backup Process Has Changed

A big problem with the dump/tar method of making backups is that you can't rely on them if the
disk is in-use.  Fixes can be tricky depending on what's going on.  If you're backing up a
database, you can dump it first and only back up the dump, not the database files themselves.
For mail spools, you might want to acquire the necessary locks when reading the file so that
you don't backup a partially-delivered mail.  Etc.  In some cases LVM snapshots might help,
but you still might take a snapshot at a very unlucky time.

Of course, none of that's a problem if you can just take the disk offline or make it read-only
before backing it up.

Personally I would use BackupPC.  http://backuppc.sf.net/

But then again I am backing up multiple computers.

This recently came up on another list I'm on. One of the answers is rsync, 
which this author didn't mention. I don't know why. Rsync can do a 
file-by-file backup, omit certain files, and a variety of other things. 
But the best part is that it only copies changed bits/changed files. And 
it will "prune" files on the backup which are no longer resident on the 
machine being backed up. It doesn't require root privileges, and it can 
use ssh or rsh as a remote shell if needed. If you need to do a bit-by-bit 
copy of a partition, no, it won't work. But otherwise it seems the perfect 
solution for most backup needs. My cron'd rsync backups are far more 
reliable than my wife's backups on her Windows machine.

I'd add a mention of DAR here (http://dar.linux.free.fr).  Designed as a Disk-ARchiver rather
than tape, it incorporates many of the features that one needs when doing backups -
compression, encryption, slicing, backup of SELinux and other extended attributes,
differential and incremental backups, and a suite of file selection methods.  It keeps a
catalogue of the files and their locations, so the disk seeks to the exact file you want
quickly, and the catalogue can be separate for incremental or differential backups relative to
fixed media (e.g. DVDs).  It may be counter to the old Unix philosophy of combining small
tools into a larger solution, but it avoids the deficiencies in interaction between those
parts.

It's dependencies are relatively small, it compiles on all architectures AFAIK and it produces
a dar_static file that can be easily put onto your backup media just in case...

I wrote a set of articles for a friend of mine about a year ago, and people have told me I
should submit them for publication somewhere.  I wonder if anyone would care to take a look at
http://sitaramc.googlepages.com/mirror-sync-backup-tools-... and comment on them?  I'd feel
a lot better about submitting them if someone who didn't know me said they were worth it ;-)

Please keep in mind that the series was written over a year ago!

Thanks,

Sitaram

Hi Forrest,

why do you use bzip2? It's much too slow to be practical. From my limited point of view, I
know nobody who'd use bzip2. gzip is the better choice.

I someone wants a very efficent, secure backup to a hard disk (possibly of several computers),
one should look at BoxBackup (http://www.boxbackup.org/). It stores a complete revision
history of every file so in theory it should be possible to restore an arbitrary point in
time, although the client currently is missing this feature.

I just published an updated version of my article about it (in German) on
http://www.pro-linux.de/berichte/boxbackup.html

Regards,
hjb

One annoying thing with backup is that it takes time. I use a rsync script with the options
--backup and --backup-dir to do incremental backups. I've added it to my /etc/rc0.d directory
so that it is executed at shutdown, I can let the script run, backup my local data and remote
server data and it shutdowns nicely when all is done.

I'll put in a plug for Dirvish:

http://www.dirvish.org/

Uses rsync, hard links to efficiently keep multiple versions of a backup set on a single
volume. I use it to back up to hard drives that I can remove and store off-site. Here's a
convenient "mobile rack" for sata drives:

http://www.addonics.com/products/mobile_rack/aesnapmrsa.asp

> Using rsync the way you've described is fine for mirroring your
> system's current state.  But it only gives you one state of the
> filesystem.  How do you recover a file that you deleted/modified but
> only noticed after a few backup cycles?
>
> If that is a requirement, then plain rsync probably isn't enough.

Yes, it is. However, there are several backup solutions which use rsync 
under the hood, one of them being rsnapshot (http://www.rsnapshot.org/).

The big advantage of using rsync is that restore is very easy (cp, scp or 
even rsync). Oh, and in case it matters, rsync can backup ACLs, which - 
to my knowledge - tar still can't handle.

Bye...

     Dirk

Anyone else using subversion/some other version control system ?

After reading a couple of articles on the subject, I put my home directory into subversion at
the beginning of last year. I do a subversion dump every night, and back up the dumps to DVD
periodically (you can do it as often as you like). I actually use a rotating set of DVD-RWs,
writing them to DVD-R a few times a year.

Subversion manages the modified/deleted files problem, and works well with binary files. It
doesn't work too well for music/videos since it keeps a second copy of your data in the .svn
directory, but rsync/tar/DVD backups take care of those.

A friend and I are both relatively well-connected by the same ISP so we wanted to cross-backup
to each other's houses.

The requirements were:
 - Automated
 - Preservation of older versions
 - Encrypted
 - Gentle to the network
 - Able to back up large amounts of data

The nasty one to find was the encryption solution, but Box Backup (http://www.boxbackup.org/)
has been performing decently well.

Dirvish looked interesting, but the backup runs appeared to need a large amount of disk or
memory to run.  Please correct me if I am wrong.

-ben

hi

I use dump for backups of my PC and co-workers. (I have developed a long and complex shell
script that dumps , incrementally, multiple partitions to a remote host.)

I dont like the proposed line
  /sbin/dump 0ufa - / | bzip2 > ./localslash`date +%Y%m%d`.bz2

In my backup script I instead  use a line as   
  /sbin/dump -0 -j7 -q -u -f - /  >   ./localslash`date +%Y%m%d`.bz2

A good point of using 'dump -j7' is that it compresses files separately (as 'zip' but unlike
'tar cjf') so that, when needed, 'restore -i' can recover single files fast and conveniently.

All these tools have been around for 20 years or more, the concept is more than 40 years old.
Time for something actually modern? I am afraid the world has changed a bit more than
reported.

Apparently the latest mac os to be released soon has continous backup: install a disk as
backup and the os keeps logging file changes to it. As reported by a friend who tried a beta,
the design support connecting the backup disk only intermittently.

Makes alll these daily, weekly and monthly rotations sound like stupid, don't you think?

Thanks to Apple for reminding us that there is no need for backups to be a sweaty process.
Computers can do it better.

The dump and tar examples are presented as if they did the same thing.  But my understanding
is that the dump example will dump the / filesystem whereas the tar example will dump all
filesystems mounted on or below /.  GNU Tar has the "--one-file-system" option that promises
to deliver the dump behavior.