User: Password:
|
|
Subscribe / Log in / New account

ext4 and data consistency

ext4 and data consistency

Posted May 14, 2010 19:15 UTC (Fri) by nix (subscriber, #2304)
In reply to: ext4 and data consistency by njs
Parent article: The Next3 filesystem

That's why robustness is so hard -- it's almost impossible to test. That doesn't mean it isn't important. All it takes is one power failure with just the right timing to trash a datastore.
Virtualization and CoW should have made this much, much easier to test in a finegrained fashion; halt the VM you're using to do the testing, CoW the file, start a new VM using the CoWed copy and mount it; note if it failed and if so how, kill the VM, remove the CoWed copy of the file and let the VM run for another few milliseconds (or, if you're being completely pedantic, another instruction!)


(Log in to post comments)

ext4 and data consistency

Posted May 14, 2010 19:36 UTC (Fri) by njs (guest, #40338) [Link]

That's a neat idea. I don't think we have cycle-accurate VMs in FOSS yet, but it doesn't matter for this, you can do the halt/check after every disk write, not every instruction. It still doesn't solve a major part of the problem -- you also need to exercise all the weird corner cases that only arise under certain sorts of memory pressure, or what happens if the disk is fragmented in *this* way and has *this* queue write depth and that makes the elevator algorithm tempted to reorder writes in an unfortunate way, etc. -- but it'd be really useful!

ext4 and data consistency

Posted May 14, 2010 20:41 UTC (Fri) by nix (subscriber, #2304) [Link]

I don't think we have cycle-accurate VMs in FOSS yet
They just need to be accurate enough that stuff works. We're not trying to make Second Reality run, here. I can't think of anything that runs on Core 2 but not AMD Phenom because of differing instruction timings!
all the weird corner cases that only arise under certain sorts of memory pressure
Seems to me that the balloon driver is what we want; it can add memory to the guest on command, can't it also take it away? I don't see why we can't do an analogue of what SQLite does in its testing procedures (use a customized allocator that forces specific allocations to fail). The disk-fragmentation stuff would take a lot more work, probably a custom block allocator, which is a bit tough since the block allocator is one of the things we're trying to test!


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds