chunkfs [LWN.net]

chunkfs

Posted Apr 27, 2007 5:31 UTC (Fri) by pimlott (guest, #1535)
In reply to: chunkfs by dlang
Parent article: Filesystems: chunkfs and reiser4

I'm not sure if I followed exactly what you meant in some places, so let me try and ask you to correct me.

it's not that you avoid dong the fsck on some chunks, it's that you don't have to try and track the state of the entire filesystem at once so the check is faster.

Are you saying here that the total fsck time of all N chunks is less than the fsck time of a normal filesystem? That's probably true, but it doesn't solve the fundamental scaling problem of full partition fsck (does it?).

if some chunks haven't been modified since they were last chaned (and can therefor be clean) that just speeds up the searh

Is "chaned" meant to be "changed"? "haven't been modified since they were last changed" doesn't make sense, however. Do you mean that if a chunk has not even been touched since the last fsck, it doesn't need to be fscked again? Well ok, but I doubt that's going to be a common case. If you mean that chunks that are not dirty after a crash don't need to be fscked, that only applies to a non-journaling filesystem (does anyone still use those?).

The premise, I thought, was that the whole filesystem needs to be fscked from time to time, because stuff happens. No matter how clever you are, stopping stopping the world to check the whole partition will not scale. So on-line fsck of chunks looks like the real win to me.

(Log in to post comments)

chunkfs

Posted Apr 27, 2007 8:05 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]

sorry for the sloppy typing, I was in too much of a hurry to finish the post

yes, the time to fsck 100x10Gb filesystems is expected to be significantly less then to fsck 1x1TB filesystem

in part this is also a memory limited operation, when doing the fsck you need to remember all the files that you have seen to see if any of the others overlap it. besides reducing n during the O(n^2) portion by 100x (even though you have to do it 100 times) you also drasticly reduce the amount of ram needed, avoiding swap or other low-memory conditions

yes, you do need to do a fsch from time to time, but under a crash condition you may be able to skip doing one for the chunks that have not been changed and so are still marked clean. and yes, people do still use non-journaling filesystems. when you journal you end up doing lots of writes twice (and a lot of seeking between the journal write and the final write). if you have lots of extra disk bandwidth you may be able to afford to do this, but if you don't have the extra disk bandwidth your entire system will slow down while the journal is flushed. for some applications (data capture for example) this isn't acceptable

in addition, you can do a check of a few chunks every boot rather then doing a check of everything every 30 boots (spreading the maintinance pain over time rather then having it hit in one massive chunk)

did I do a better job of explaining it this time?