Transparent decompression for ext4

Posted Aug 1, 2013 21:59 UTC (Thu) by neilbrown (subscriber, #359)
In reply to: Transparent decompression for ext4 by dlang
Parent article: Transparent decompression for ext4

> It seems like this would be good for shared library files as well.

Except that shared libraries are memory-mapped which isn't supported yet...
And it strikes me that there would be an awful lot of complexity to go from "uncompress whole file at once" to "uncompress pages individually to support demand paging for memory mapping"

Given that this seems to be aimed at specific use cases, I think I would lean towards letting those use-cases deal with uncompression themselves.

zcat/zless/zgrep demonstrate to me that it isn't really that hard to add decompression to specific use cases.

I'm sure you could write a dlopen which did transparent decompression of libraries - cache them in ramfs and map them from there...

Transparent decompression for ext4

Posted Aug 1, 2013 22:06 UTC (Thu) by dlang (guest, #313) [Link]

when an application executes and the system loads the shared library, it doesn't just mmap misc pages, it first read enough about the library to know what is where so that it would know what pages need to be mmapped.

It seems to me that this process could be tweaked to understand this sort of transparent compression, and uncompress the entire library (or, in most cases, find where the library has already been uncompressed to support other applications)

Even if this means that the entire library must sit in ram while it's used, this won't make it worthless (it will mean that a huge library that's not used by many apps, and the apps only use a small portion of it may not be good candidates for compression)