User: Password:
|
|
Subscribe / Log in / New account

Squashfs submitted for the mainline

Squashfs submitted for the mainline

Posted Oct 30, 2008 1:18 UTC (Thu) by sbergman27 (guest, #10767)
Parent article: Squashfs submitted for the mainline

Squashfs can also make use of lzma compression which achieves an amazing level of compression (depending upon the data, of course). I believe that capability exists in 3rd party patches and is not part of the current, official SquashFS. OpenWRT uses this for their router firmware images, and what they are able to pack into a few MB is nearly unbelievable.


(Log in to post comments)

Squashfs submitted for the mainline

Posted Oct 30, 2008 9:48 UTC (Thu) by Trou.fr (subscriber, #26289) [Link]

Talking about LZMA, are there any explanations for why it is not going to be included in the kernel (it seems) ?

Squashfs submitted for the mainline

Posted Oct 30, 2008 10:02 UTC (Thu) by dlang (subscriber, #313) [Link]

I've seen some discussions about the possibility of using LZMA for the boot kernel, and one issue that I've seen is that LZMA can use incredible amounts of ram under corner cases

Squashfs submitted for the mainline

Posted Oct 30, 2008 11:43 UTC (Thu) by Tuxie (guest, #47191) [Link]

But AFAIK that depends on how the data was compressed, which parameters was used. You CAN do LZMA compression with extreme parameters which will only slightly improve compression but will take 100x the time and/or RAM usage for both encoding and decoding. Just use sane parameters when compressing the kernel and everything will be fine...

Squashfs submitted for the mainline

Posted Oct 30, 2008 11:59 UTC (Thu) by epa (subscriber, #39769) [Link]

But can you guarantee a maximum memory usage for decompressing? With gzip you can allocate a 32kbyte window and a little bit of housekeeping and you're guaranteed to decompress any valid stream. No dynamic memory allocation is needed. Can LZMA say the same?

Squashfs submitted for the mainline

Posted Oct 30, 2008 13:22 UTC (Thu) by ajb (subscriber, #9694) [Link]

If you're doing the compressing, you ought to be able to guarantee a memory limit for decompression. Just try decompressing while you still have the original around, and tweak the compression parameters if you run out of memory. That's how MPEG encoders work - consumer products that do mpeg stream decompression have a defined amount of memory (2Mbits, if I remember correctly).

Admittedly that might be considered an expensive solution, but it proves that a solution exists.

Squashfs submitted for the mainline

Posted Oct 30, 2008 13:42 UTC (Thu) by Tuxie (guest, #47191) [Link]

According to Wikipedia, "LZMA is effectively deflate (zlib, gzip, zip) with a larger dictionary size, 32MB instead of 32kB".

It also say: "the amount of RAM required during decompression is principally determined by the size of the sliding window used during compression. Small code size and relatively low memory overhead, particularly with smaller dictionary lengths, make the LZMA decompression algorithm well-suited to embedded applications."

I'm not sure if that's a good enough answer. :)

Squashfs submitted for the mainline

Posted Oct 30, 2008 9:51 UTC (Thu) by Tuxie (guest, #47191) [Link]

I was going to post about LZMA support also. AFAIK the only reason it's not part of the official SquashFS is to make the LKML beards less resistant to including it in mainline. Too bad, because it makes quite a big difference in read-performance on slow media such as CDs, NBD and USB sticks, and space on storage-starved embedded devices.

That's not the question of pointless resistrance

Posted Oct 30, 2008 10:46 UTC (Thu) by khim (subscriber, #9252) [Link]

Squashfs with LZMA support includes all this code in filesystem itself and that's "just wrong(tm)". It must be available for other pieces of kernel as well. I suppose squashfs developers can do this - but that's move patch from "yet-another-filesystem-patch" (easy to get approval and integrate) to "yet-another-patch-with-changes-to-core-API" (much harder to process). So the split is right. I sure hope developers will not stop at just adding squashfs but will proceed with LZMA support as well, but... later.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds