LWN.net Logo

And this is relevant to ext4... exactly how?

And this is relevant to ext4... exactly how?

Posted Dec 2, 2011 18:55 UTC (Fri) by walex (subscriber, #69836)
In reply to: And this is relevant to ext4... exactly how? by tytso
Parent article: Improving ext4: bigalloc, inline data, and metadata checksums

One nice thing with ext4 is that fsck times are reduced (typically) by a factor of 7-12 times. So a TB file system that previously took 20-25 minutes might now only take 2-3 minutes.

That is the case only for fully undamaged filesystems, that is the common case of a periodic filesystem check. I have never seen any reports that the new 'e2fsck' is faster on damaged filesystems too. And since a damaged 1.5TB 'ext3' filesystem was reported take 2 months to 'fsck', even a factor of 10 is not going to help a lot.


(Log in to post comments)

And this is relevant to ext4... exactly how?

Posted Dec 2, 2011 19:10 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]

I've had to do fsck on multi-TB filesystems after unclean shutdowns, they can take a long time, but time measured in hours (to a couple days for the larger ones). I suspect that if you are taking months, you have some other bottleneck in place as well.

And this is relevant to ext4... exactly how?

Posted Dec 3, 2011 0:40 UTC (Sat) by walex (subscriber, #69836) [Link]

An unclean shutdown is usually not that damaged, which can however happen with a particularly bad unclean shutdown (lots of stuff in flight, for example on a wide RAID) or RAM/disk errors. The report I saw was not for a "enterprise" system with battery, ECC and a redundant storage layer.

And this is relevant to ext4... exactly how?

Posted Dec 2, 2011 21:41 UTC (Fri) by nix (subscriber, #2304) [Link]

This has been wrong for years. As long as your filesystem was built with the uninit_bg option (which it is by default), block groups which have never been used will not need to be fscked either, hugely speeding up passes 2 and 5 (at the very least).

Fill up the fs, even once, and this benefit goes away -- but a *lot* of filesystems sit for years mostly empty. fscking those filesystems is very, very fast these days (I've seen subsecond times for mostly-empty multi-Tb filesystems).

And this is relevant to ext4... exactly how?

Posted Dec 2, 2011 22:45 UTC (Fri) by tytso (subscriber, #9993) [Link]

We could fix things so that as you delete files from a full file system, we reduce the high watermark field for each block group's inode table, which would restore the speedups caused by needing to scan the entire inode table. I haven't bothered to do this, but I'll add it to my todo list. (Or someone can send me a patch; it would be trivial to do this at e2fsck, but we could do it in the kernel, too.)

Not all of the improvements in fsck time come from being able to skip reading portions of the inode table. Extent tree blocks are also far more efficient than indirect blocks, and so that contributes to much of the speed improvements of fsck'ing an ext4 filesystem compared to an ext2 or ext3 file system.

And this is relevant to ext4... exactly how?

Posted Dec 2, 2011 23:35 UTC (Fri) by nix (subscriber, #2304) [Link]

We could fix things so that as you delete files from a full file system, we reduce the high watermark field for each block group's inode table
That seems hard to me. It's easy to tell if you need to increase the high watermark when adding a new file: but when you delete one, how can you tell what to reduce the high watermark to without doing a fairly expensive scan?

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds