Why kernel.org is slow: EXT3 file fragmentation? [LWN.net]

Why kernel.org is slow: EXT3 file fragmentation?

Posted Jan 11, 2007 11:38 UTC (Thu) by etienne_lorrain@yahoo.fr (guest, #38022)
Parent article: Why kernel.org is slow

> To make things worse, while the ext3 filesystem tries hard to lay out
> files contiguously on the disk, it does not make the same effort with
> directories.

It is maybe only me, but I have a problem with creating contigous
files on EXT3, for quite some time.
I need them for my Gujin bootloader, to simulate a floppy/hard disk
before booting, a small resident software simulating a BIOS disk
to do all sort of stuff. The resident reads the data from disk, so do
not manage holes at all for technical reasons, and while it is easy
to create contigous files with ISO9660 filesystems where all files
are contigous, FAT and mostly EXT* are more a problem.

Even if you submit the complete file to write at once by low-layer
interface of Linux, you often end up with multiple blocks - maybe
linked to some security or load balancing setup.

I have even written two small program if someone wants to experiment,
one to copy a file trying to get a single segment (and so a 12 blocks
hole at its begining for EXT2/3), retrying multiple times if it does
not achieve it, and one to display the file segments on the partition/disk.

To reproduce, get gujin-1.6.tar.gz from http://gujin.org or sourceforge,
gunzip/untar and, choosing a big file like "disk.c" from an ext2/3
partition (YMMV) :
# make showmap addhole
# su
# ./addhole disk.c holly_disk.c
Warning: created file with multiple segments, renaming to 'htmpA' and retrying
Warning: created file with multiple segments, renaming to 'htmpB' and retrying
Warning: created file with multiple segments, renaming to 'htmpC' and retrying
Success: cleaning intermediate files
# ./showmap holly_disk.c
File "holly_disk.c" of size 306600 (512 blocks512) is on filesystem 0xFE0A.
Device block size: 512, FS block size: 4096, device size: 41943038 blocks
Device length: 21474835456 bytes
The device start at 0, C/H/S: 0/0/0.
File (75 blocks of 4096) begin with a hole of 12 block, and start at block 334377 for 63 blocks,
last block 334439 and file has 1 fragments.
FIBMAP succeeded after end of file, block index 75 give block 0
# rm holly_disk.c

Maybe time for an EXT2/3 analyser/defragmenter, or to disable something somewhere?

to post comments

Why kernel.org is slow: EXT3 file fragmentation?

Posted Jan 13, 2007 22:13 UTC (Sat) by jzbiciak (guest, #5246) [Link] (1 responses)

The EXT2/EXT3 filesystem has a number of structures, such as block group bitmaps, that occur in a regular pattern over the surface of the disk. While ext2/ext3 try to minimize gaps in a file, these fixed structures do interrupt large files. So, in your case, you're probably hitting that. Furthermore, no amount of allocator smarts will make a fully contiguous file if it's larger than the distance between two block bitmaps.

Note that fragmenting a file in this manner tends not to be a performance issue since large linear reads still mostly read the file of interest, and skipping a few blocks doesn't typically cause a seek. That is why I imagine it's not widely considered an issue. It's quite a bit different than, say, the old MS-DOS first-fit policy that could cause a file to be literally spread piecemeal across the entire filesystem. That sort of fragmentation destroys performance.

Why kernel.org is slow: EXT3 file fragmentation?

Posted Jan 15, 2007 11:51 UTC (Mon) by etienne_lorrain@yahoo.fr (guest, #38022) [Link]

No, here as shown in the small and quick example, to create a contigous file of approx 300 Kbytes, the small software had to try 4 times; the first 3 times the file was not contigous.
The first 3 files are not deleted but renamed after each unsuccessfull tries, to not try to position the new file at the exact same place on the disk.
I'd bet there has not been even one "EXT2/3 fixed area" collision, those shall be very unusual, and the behaviour I see on multiple distributions is very usual - the small software stops trying after 16 unsuccessfull (i.e. non contigous) files because sometimes I got 10 non-contigous 1.44 Mbytes files (unloaded machine, plenty of space on the FS)...
I do agree on the fact that if the two fragments are nearby it should not produce performance loss.