Next-generation Linux file systems: NiLFS(2) and exofs (developerWorks) [LWN.net]

Next-generation Linux file systems: NiLFS(2) and exofs (developerWorks)

Posted Nov 12, 2009 4:29 UTC (Thu) by Simetrical (subscriber, #53439)
In reply to: Next-generation Linux file systems: NiLFS(2) and exofs (developerWorks) by johnflux
Parent article: Next-generation Linux file systems: NiLFS(2) and exofs (developerWorks)

1) More than that: writing a file and then changing one block (or appending
a block) will put different parts of the file in totally different places.
A log-structured filesystem is more or less guaranteed to be 100% fragmented
for files larger than a few blocks or files that change often. This is why
it's really only suitable for SSDs, where fragmentation borders on
irrelevant.

2) Log-structured filesystems devote much ingenuity to garbage collection.
They delete or reposition old entries intelligently as needed. Either the
file will be skipped, or moved ahead. (I would assume skipped, but I don't
know much of anything about log-structured filesystems.)

(Log in to post comments)

Next-generation Linux file systems: NiLFS(2) and exofs (developerWorks)

Posted Nov 19, 2009 15:46 UTC (Thu) by forthy (guest, #1525) [Link]

Actually, optimizing for read is dead easy. Log structured file systems are by nature also copy on write, so just read the files in and write them ("in place") out in the order you want them to be read (e.g. the files loaded at boot or login time). Result: All nicely defragmented and in the right order so that readahead works perfectly.

The main recipy to defragmentation is delayed allocation, anyways, and that works the same for log structured as for ext3. The files that are going to end up pretty fragmented are logs, or similar files where processing takes long and which are written in smaller chunks. These files have to be cleaned up in the collecting process.

A log structured file system that keeps kryptographic strong checksums/hashes for all files can easily remove redundand copies by looking at the hashes only. When idle, it can copy the most fragmented files and the ones from sparcely used areas of the disk, to get free space. Keeping strong hashes is only easy for append-only files, so randomly written files like database files still won't be compressed.