| |||||||||||||
Interesting work - and some ideas for the futureInteresting work - and some ideas for the futurePosted Jul 6, 2006 9:55 UTC (Thu) by nix (subscriber, #2304)In reply to: Interesting work - and some ideas for the future by ayeomans Parent article: The 2006 Linux Filesystems Workshop (Part III)
I have a number of ideas on the versioning filesystem front which are congealing into a design. Eventually people will stop giving me new ideas faster than I can figure out how to use them :) the tree-of-blocks-CoW stuff is a critical part of it, of course.
The block allocation stuff isn't the hard part (we can start with something naive and get more complex later on): the optimizers are the hard part, because we *really* want to share as much data as possible, not least because my chosen semantics for versioning involve a new version on every close() of an open-for-writing file and on every update of a directory. This also means we need automatic expiry.
(It's also versioning both by name and by inum --- essential if it's to be useful with most text editors --- which interacts interestingly with hard links and the permissions system.)
(Because it's very much a prototype and because the data structures are not simple I'm initially prototyping the thing inside PostgreSQL, with access via FUSE. Figuring out how to turn the whole thing into a more conventional filesystem can wait. What's that about typical impractical researcher's mentality? OK, OK, I admit it ;) )
(Log in to post comments)
Interesting work - and some ideas for the future Posted Jul 13, 2006 19:36 UTC (Thu) by ringerc (subscriber, #3071) [Link] Versioning FSs get problematic when you consider that they're most useful in areas like network file servers and home directories. Lots of programs like to drop large amounts of crap in these places, much of which should not be versioned. Consider program scratch files, thumbnail DBs, etc. Identifying these files and avoiding versioning them would be extremely useful.
Ageing out versions sounds like a good idea. You might also want to look into a winnowing process where fewer versions are kept further back in time, rather than just using a strict version count or time limit. I refer to something akin to the way round robin databases work - you keep one file from a month ago, one for each prior week, one every day for this week, and one every hour (assuming of course that the file was actually modified in each period). This helps reduce the "damn, I save every five minutes so my versions only go back an hour" problem.
Being able to mount a read-only view of the FS frozen at a point in time would be an incredibly nice inteface to the versions that wouldn't require any special tools. User says "Yeah, I deleted it some time today" ... so I just:
and the user can just access `yesterday's files' with samba. For that matter, imagine a rolling view:
mount -o version_age=24h ....
so the mount shows files as they were 24h ago, updating (roughly, presumably in chunks) with time. Stupid idea? Probably. Useless? Probably. Fixed point-in-time views that were mountable would be amazingly useful though ... like a coarse continual snapshot.
Interesting work - and some ideas for the future Posted Jul 15, 2006 11:23 UTC (Sat) by nix (subscriber, #2304) [Link] An avoidance list will be mandatory: probably one global one and one per-directory (non-inherited). We don't want to version vim .swp files, for starters :)
The winnowing idea sounds excellent: I'll incorporate it if I can figure out a user interface! (It sounds like the sort of thing which should be controlled by a minimal cutoff date and some sort of logarithmic or exponential parameter.)
As for the read-only view, well, the current design already has the ability to roll individual files, directories, and trees backwards and forwards in time. It's not stupid at all. :)
Oh, and it's not read-only. If you write to a file (or modify a directory) which is rolled back into the past, you get a branch.
I think I've managed to conform to POSIX everywhere: I'm in the middle of verifying this at the moment. From the point of view of apps which have files open when someone rolls them back, it just looks like someone's done a truncate or a big write... the whole point of this is POSIX conformance with version control layered on top: I don't want to turn out yet *another* versioning system which doesn't support hard links, symlinks, or permissions!
|
|||||||||||||