Filesystem defragmentation [LWN.net]

Filesystem defragmentation

Posted Mar 26, 2015 16:41 UTC (Thu) by pr1268 (guest, #24648)
Parent article: Filesystem defragmentation

Forgive me for being a little unenlightened, but didn't I read somewhere a number of years ago that Linux filesystems (e.g. Ext2/3/4)¹ are resistant to fragmentation (or at least the ill effects thereof) by design?

Not that I'm trying to dismiss Dmitry's work. I suppose SMR devices expose new engineering challenges with respect to filesystem layout.

Also, just an idea: If Dmitry's utility is as filesystem-agnostic as mentioned, why not drop the "e4" from the tool's name. Or, how about a more universal-sounding "fsdefrag2"? (Again, just a suggestion—I'm pleased that someone is working on keeping our filesystems neat-and-orderly.) :-)

¹ Also ReiserFS, JFS, XFS, etc.

to post comments

Filesystem defragmentation

Posted Mar 26, 2015 22:45 UTC (Thu) by flussence (guest, #85566) [Link]

Linux filesystems *are* resistant to fragmentation, in the sense that the allocation algorithms make an effort to find contiguous spans to write new data to, while avoiding packing files like sardines in case some of them need to be appended to.

There's still good reasons for defragmentation, the main one being the above scheme tends to fragment free space over time, making it increasingly difficult to find large contiguous areas on a disk that's filling up. There's also the use case of boot and possibly individual applications, where you can measure an access pattern once and optimize heavily for it - `e4rat` does that, but the name implies it's ext4-only.

The official builds of Firefox have been doing a similar thing for a few years now, by packing all the data needed for startup into a large, carefully crafted .so file that causes it to be read more or less linearly. If some of that behaviour can be done in the filesystem, everyone wins.

Filesystem defragmentation

Posted Mar 30, 2015 4:58 UTC (Mon) by martinfick (subscriber, #4455) [Link] (1 responses)

I think this myth is so prevalent all over the internet, that people now just assume it is true. Try downloading the android source tree using repo -j 24 and I suspect you will notice a 10-20% degradation in copying that tree, right of the bat, with respect to a tree that was synced with a -j 1. Even a -j 8 suffers noticeably I think. Add in builds mixed with resyncs (from yan upstream which gets lots of regular little changes) and the directory tree will eventually become unusable (with ext4 at least). It is great that people are starting to recognize some fragmentation issues. I hope that work can also be done on the FSes themselves to improve this so that a separate defragmentation tool is not needed.

Filesystem defragmentation

Posted Mar 30, 2015 13:23 UTC (Mon) by Jonno (guest, #49613) [Link]

> I think this myth is so prevalent all over the internet, that people now just assume it is true.

Well to be honest there is *some* truth to this "myth", but fragmentation resistance is not a binary thing, but a matter of degree, and while Linux is a whole lot better than Windows, it is not (and can not possibly be) perfect.

It's a bit like the FAT vs NTFS story back in the NT4 days. NTFS was so much better than FAT that MS thought a defragmentation tool in NT4 was unnecessary. However, after several months of typical use (or several days of a pathologically bad use pattern) the performance penalty still grew significant, and MS had to back-pedal.

Now the Linux file systems are even better than NTFS, but the story is essentially the same, except knowledgeable Linux people have generally not claimed defragmentation tools to be completely unnecessary, only not a priority (until now, apparently).