Backing up in trees with Obnam 1.0
Posted Jun 7, 2012 8:05 UTC (Thu) by oever
In reply to: Backing up in trees with Obnam 1.0
Parent article: Backing up in trees with Obnam 1.0
Yes it does. LWN has written wrote about bup. The bup README document is a great read. It explains that while bup is similar to git in that it uses Merkle trees, but also how it is different because it splits up big files by, like rsync, using a rolling checksum. By doing so, deduplication works better than when using fixed blocks.
Imagine a large random file to which you prepend one byte. To a block-based deduplication algorithm, the entire file has changed and there is no deduplication happening. With a rolling checksum method, the first block is different but all subsequent blocks are the same. This method of deduplication is mainly useful for backing up filesystems and databases efficiently, but also helps backing up compressed archives such as zip files (but not or less for compressed tar files).
Using a rolling checksum for doing backups, like bup does, is genius. As far as I can tell, neither ocnam, tarsnap or ddar use a rolling checksum for deduplication.
to post comments)