| |||||||||||||
Google releases a better compression algorithmGoogle releases a better compression algorithmPosted Mar 1, 2013 1:15 UTC (Fri) by wmf (guest, #33791)In reply to: Google releases a better compression algorithm by jke Parent article: Google releases a better compression algorithm
pigz and pxz exist; I don't know if they've been librarified but it would seem the hard work has been done.
(Log in to post comments)
Google releases a better compression algorithm Posted Mar 1, 2013 1:45 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]
Mock, the tool used by the Fedora build system as well as packagers for building packages in a chroot uses pigz based on my suggestion and it seems to work fine for our purposes.
Google releases a better compression algorithm Posted Mar 1, 2013 1:57 UTC (Fri) by keeperofdakeys (subscriber, #82635) [Link]
Pxz has its own disadvantages though. Since (from what I have read) LZMA can't be multithreaded easily, pxz compresses different parts of the file at the same time (dumping them in /tmp), and finally putting them all together at the end. For solid archives this reduces the compression ratio in exchange for a (quite significant) speedup. In most cases the ratio watsed is quite small, so the tradeoff is worth it.
Google releases a better compression algorithm Posted Mar 1, 2013 9:16 UTC (Fri) by nmav (subscriber, #34036) [Link]
Indeed. However I'd suggest looking at plzip instead of the xz tools. I was surprized by the difference in compression ratio between them, even if they are based on the same compression algorithm.
http://www.nongnu.org/lzip/lzip.html
Google releases a better compression algorithm Posted Mar 1, 2013 8:02 UTC (Fri) by jezuch (subscriber, #52988) [Link]
There's also pbzip2. Bzip2, being a BWT-type algorithm, is quite easily parallelizable with little cost to compression ratio.
Google releases a better compression algorithm Posted Mar 2, 2013 5:06 UTC (Sat) by SEJeff (subscriber, #51588) [Link]
pbzip2 is incredible for bz2 files, but it will use up 100% of your available memory when you run it on big binary files. even with ionice -c3, it can be very intrusive
|
|||||||||||||