ext3 defragmentation

Posted Apr 22, 2004 16:42 UTC (Thu) by Duncan (guest, #6647) [Link]

There's not enough info in that limited quote to tell for sure, but if an
ext2 defragger is run on an ext3 filesystem either mounted, or unmounted
but with unsynced data in the journal, it WILL likely corrupt data,
because the ext3 side of things won't be aware of the data-blocks moving
out from where it expects them to be, and could easily attempt to write
data to the OLD location.

Some years ago, back in early '98, b4 I switched from MSWormOS and while I
was participating in the public betas for IE4, it had a similar conflict
between the IE cache code and the 95 defragger. IE3 had used a mechanism
where by the cache index file location was kept in memory for direct
writing, bypassing the file-system lookup after it was loaded and writing
directly to disk, for performance reasons. That worked with IE3, because
it was only temporarily loaded and was shut down during defrags. MS
changed the rules with IE4 and its desktop extensions, however, and it
remained loaded as long as Windows Explorer was loaded, because it WAS now
Windows Explorer as WELL as IE. Normally, such constantly loaded "system"
files remained untouched and unmoved by the defragger. Unfortunately in
the IE4 second beta, they forgot to set the IE cache index file as
"system" and the defragger would move it out from under the still live
IE/WE process, which would then write all over whatever replaced the file,
when it tried to rewrite it to disk. The simple enough fix was to set the
system flag on the file, but IE would reset it every time it was started,
so one had to keep on top of things.

Fortunately, here, I'd set up my temp dir as a seperate partition, and had
the IE cache set to use my temp partition. Thus, the only data I had
exposed was temporary anyway, and the bug wasn't a big problem for me.
However, some of the other beta newsgroup regulars and others that posted
only when they had problems, lost valuable data to that bug. Keeping temp
files in their own partition saved my butt, but I've never forgotten that,
as it left a BIG impression on me, not only on the risks of beta, but ALSO
on the wisdom of limiting potential damage with multiple partitions. Of
course, it also impressed me with the wisdom of making SURE nothing is
going to be writing to that defragged partition without being aware of the
new location of the data.

It's entirely possible the test was done with a properly unmounted and
journal-empty ext3 partition for the defrag, but if it wasn't, it's no
surprise there were data integrity issues.

Duncan

Posted Apr 23, 2004 1:43 UTC (Fri) by prat (guest, #20866) [Link]

I've actually looked at this before, and found a few solutions, but quickly concluded that the programs in question (which had to be run offline) seemed a little too unmaintained and unreliable to test out on my partition. The best answer I've gotten from anyone so far is that yes, there is a program that can do this, and that program is tar. =) Of course, you'd need someplace to back everything up to. Then you can just untar everything onto a clean partition, possibly with these new patches in place in the kernel you use during the restore, but since tar is probably doing this linearly anyway, I doubt it would make much difference.

Long story short, most people I've talked to have never had any problems with fragmented ext[23] filesystems. Sorry.

Posted Apr 25, 2004 11:15 UTC (Sun) by tialaramex (subscriber, #21167) [Link]

ext3 filesystems that are cleanly unmounted are also valid ext2 filesystems and can therefore be defrag'd by any working ext2 defrag application. I last used such a thing in 1998 and it seemed to work fine at that time. I believe there have since been one or perhaps at most two incompatible changes to the ext2 structures that might (not sure) affect defrag programs. You should check that any defrag application you consider is aware of ext2/3's compatibility flags, which would prevent it from modifying a filesystem with features that it doesn't understand.