How The Backup Process Has Changed [LWN.net]

How The Backup Process Has Changed

Posted Nov 29, 2007 17:43 UTC (Thu) by Los__D (subscriber, #15263)
In reply to: How The Backup Process Has Changed by hjb
Parent article: How The Backup Process Has Changed

If you need speed, go with gzip (or maybe even just plain tar), if you need space, go with
bzip2...

(Log in to post comments)

How The Backup Process Has Changed

Posted Nov 30, 2007 3:59 UTC (Fri) by sitaram (subscriber, #5959) [Link]

I have this bad habit of doing all my research in one shot, as exhaustively as I can,
summarising it for quick reference, and then discarding the raw data :-)

With that caveat, here is a dump of the entry titled "Choosing between LZMA, BZIP2, and GZIP"
in my personal quickref wiki:

---------------------------

LZMA is the new kid on the block: less space and faster decompression (than
BZIP2) at the cost of much, *much* slower compressions.

(Default compression levels are GZIP: 6, BZIP2: 9, and LZMA: 7)

Summary
-------

    Use none when
      - almost all files in the dataset are already compressed (DUH!)

    Use GZIP when
      - time is more important than space, or
      - system memory is very limited, or
      - a lot of files in the dataset are already compressed but nowhere near
        all of them

    Use LZMA when
      - space is more important than time, or
      - space is important AND the file will be decompressed many times

    Benchmark BZIP2, LZMA at level 1 and perhaps LZMA at level 2 when
      - both space AND (compression) time are important, and
      - you're going to be compressing this same dataset frequently (like a
        daily backup script for your email folders)

    Otherwise just use BZIP2

How The Backup Process Has Changed

Posted Dec 8, 2007 1:52 UTC (Sat) by roelofs (subscriber, #2599) [Link]

Use none when
- almost all files in the dataset are already compressed (DUH!)

Major omission, both here and in the main article: also use none when your backup medium (and/or the path to it, including RAM) may have errors. Both compression and encryption largely destroy any ability to recover data past the error location. (I discovered two bad bits in 1 GB of memory while verifying a backup to DVD+R.)

Otherwise just use BZIP2

bzip2 is much, much slower than gzip on decompression, too. If it's read-once (or read-none), then that may not matter. But for read-many it's pretty bad. (I have no data on LZMA or other alternatives. Capacity is cheaper than CPU, however.)

Greg