Why kernel.org is slow: EXT3 file fragmentation?
Why kernel.org is slow: EXT3 file fragmentation?
Posted Jan 13, 2007 22:13 UTC (Sat) by jzbiciak (guest, #5246)In reply to: Why kernel.org is slow: EXT3 file fragmentation? by etienne_lorrain@yahoo.fr
Parent article: Why kernel.org is slow
The EXT2/EXT3 filesystem has a number of structures, such as block group bitmaps, that occur in a regular pattern over the surface of the disk. While ext2/ext3 try to minimize gaps in a file, these fixed structures do interrupt large files. So, in your case, you're probably hitting that. Furthermore, no amount of allocator smarts will make a fully contiguous file if it's larger than the distance between two block bitmaps.
Note that fragmenting a file in this manner tends not to be a performance issue since large linear reads still mostly read the file of interest, and skipping a few blocks doesn't typically cause a seek. That is why I imagine it's not widely considered an issue. It's quite a bit different than, say, the old MS-DOS first-fit policy that could cause a file to be literally spread piecemeal across the entire filesystem. That sort of fragmentation destroys performance.
Posted Jan 15, 2007 11:51 UTC (Mon)
by etienne_lorrain@yahoo.fr (guest, #38022)
[Link]
No, here as shown in the small and quick example, to create a contigous file of approx 300 Kbytes, the small software had to try 4 times; the first 3 times the file was not contigous.Why kernel.org is slow: EXT3 file fragmentation?
The first 3 files are not deleted but renamed after each unsuccessfull tries, to not try to position the new file at the exact same place on the disk.
I'd bet there has not been even one "EXT2/3 fixed area" collision, those shall be very unusual, and the behaviour I see on multiple distributions is very usual - the small software stops trying after 16 unsuccessfull (i.e. non contigous) files because sometimes I got 10 non-contigous 1.44 Mbytes files (unloaded machine, plenty of space on the FS)...
I do agree on the fact that if the two fragments are nearby it should not produce performance loss.