Recent Features
| |||||||||||||
Soft updates, hard problemsSoft updates, hard problemsPosted Jul 3, 2009 21:16 UTC (Fri) by jzbiciak (✭ supporter ✭, #5246)In reply to: Soft updates, hard problems by jzbiciak Parent article: Soft updates, hard problems
An LJ user pointed me to some very interesting work in this area. He wrote the following: The Featherstitch filesystem uses clever formal systems techniques to derive softupdates ordering requirements and optimise away unnecessary disk operations. They've basically generalized and automated Kirk McKusick's one-off analysis of FFS's ordering requirements. This technology means softupdates is no longer difficult. I thought I should share it over here.
(Log in to post comments)
Soft updates, hard problems Posted Jul 3, 2009 21:27 UTC (Fri) by vaurora (guest, #38407) [Link]
Yes, I think Featherstitch is very interesting - I saw the paper when it was first presented and was impressed. Would anyone be interested in reading an LWN article about it?
Soft updates, hard problems Posted Jul 3, 2009 21:47 UTC (Fri) by jzbiciak (✭ supporter ✭, #5246) [Link]
Put my vote in the "Yes" column. :-)
Soft updates, hard problems Posted Jul 4, 2009 2:39 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]
definantly yes.
it would either doucment a tool that could be used by kernel developers, or give us answers for the (now inevitable) swarm of questions about why don't the kernel devs just use featherstitch and implement soft updates ;-)
is there a filesystem simple enough to show examples of it's analysis? (I'm thinking either ext2 of ext4 without journling as possibilities)
Featherstich Posted Jul 4, 2009 5:16 UTC (Sat) by ewen (subscriber, #4772) [Link]
Yes, I'd be keen to see an article on Featherstich. The article on soft updates was good to see here too; I'd read the original paper on soft updates and admired the approach taken even though it seemed to have a "Level of Difficulty == Superhuman". But there are many problems in computer science like that (write a modern operating system in assembly language, anyone?), for which the computer science answer is another layer of abstraction (and tools to translate back to the lower levels). It feels like that's what we need here too: a way to express the dependency graph of all the operations such that both the code to do it, and the code to track the order in which things hit disk, is auto-generated.
Ewen
Soft updates, hard problems Posted Jul 4, 2009 0:20 UTC (Sat) by njs (guest, #40338) [Link]
Huh, and what's *really* cool about it is that they apparently export a userspace API for expressing IO write barriers. So with the featherstitch patch, you really can say things like "write these bytes to that file, and then rename it, but don't let the rename hit the disk until after the write has completed", except with arbitrarily complex dependency structures (!).
I don't really care whether I'm using soft-updates or journalling, just so long as my disk is safe and things are reasonably fast, but real write barriers could totally change the performance profile of higher level apps. (Except for ACID databases, of course; I guess that's why historically no-one has cared about write barriers, because the only important high-level storage service was ACID databases.)
Soft updates, hard problems Posted Jul 4, 2009 0:41 UTC (Sat) by njs (guest, #40338) [Link]
They got a bunch of the important stuff right, too... the "patchgroup" handles are basically fd's, they correctly prevent circular dependencies, they do the Right Thing for all different modes of fs operation (soft-updates: barriers use soft-update magic, full journaling: barriers optimize out, metadata journaling: barriers magically upgrade only those operations with dependencies to use full journaling), and check out this example from the paper:
> The following command line would ensure that 'in' is not removed until all changes in the preceding sort have committed to disk: $ depend 'sort < in > out' 'rm in'
(Of course, really you want shell support, so you can just say 'sort <in >out &&! rm in' or whatever.)
Aw, man, now I'm all excited about a research project from 2007 that will never go anywhere in real life :-(.
Soft updates, hard problems Posted Jul 4, 2009 12:06 UTC (Sat) by nix (subscriber, #2304) [Link]
Seconded. This looks really nice...
|
|||||||||||||