Embedded Linux: Using Compressed File Systems
Posted Feb 8, 2007 17:52 UTC (Thu) by moxfyre
In reply to: Embedded Linux: Using Compressed File Systems
Parent article: Embedded Linux: Using Compressed File Systems
I know of only two filesystems that are both writeable and compressed: jffs2 and e2compr. jffs2 works only on directly-connected flash (i.e. not the USB and IDE flash drives that I use), and IIRC e2compr is not included in the kernel because it doesn't have any fsck support, which is rather worrying to say the least. The remaining option is to use a union filesystem to combine a read-only compressed filesystem with a writeable uncompressed filesystem. This is fine at first but becomes useless once you've "apt-get upgrade"ed - eventually it uses more space than a normal uncompressed filesystem would have done.
So there's definitely scope for improvements. There are other things you could do, such as initially storing the data uncompressed and having a low-priority background process do the compressing once files seemed to be stable. Maybe this could happen as part of the journal writeback system?
I agree completely.
The lack of transparent r/w compression is a frustrating situation, especially for those who want to make Live USB Flash distributions. For a Live CD, you can just use squashfs with unionfs... since the media isn't easily rewritable. But with a space-constrained installation on a USB Flash disk, I'd like to have transparent read/write support AND compression. Something like what "dblspace" or "Stacker" did under MS-DOS.
The best-known project to do this under linux, e2compr, is basically moribund. I think there has not been much demand for such a filesystem for two reasons:
- Hard drives have gotten bigger so quickly that there's little need to compress most of the files on a desktop or laptop computer.
- For those files that *do* take up a lot of space (audio, video, source code archives) there are domain-specific file-level compression algorithms that give much better performance than a general-purpose solution.
I imagine that the widespread use of flash-based storage will soon bring back the demand for good, transparently compressed storage. Here are a couple promising projects to add such a filesystem to Linux: compFUSEd
to post comments)