| |||||||||||||
snapshots?snapshots?Posted Jan 26, 2006 20:14 UTC (Thu) by nix (subscriber, #2304)In reply to: snapshots? by xav Parent article: MD / DM
If you don't mind storing your filesystem in PostgreSQL and accessing it via FUSE, I'm on it. File-by-file and directory-by-directory rollback-and-forward, with branching (roll back and write). A `commit' is done on every open for writing; versioning is done both by filename and by inode number, so editors that write files by unlink()-and-rename() are covered.
(The overhead is necessarily considerable, although access to data at branch tips, which should be most of the accesses is still O(1).)
I'll admit it's mostly for the fun of it... I should have a trac up with design docs and a public svn repo in a week or so (hardware replacement at this end first so I've got the disk space to play with things like this!) and be working on the actual code.
(Log in to post comments)
snapshots? Posted Jan 27, 2006 0:16 UTC (Fri) by drag (subscriber, #31333) [Link] There is those "log-structure" filesystems. There are a couple currently in the works for Linux.. one from a telecom company from Japan and another one that made it into that 'google summer of code'.
http://logfs.sourceforge.net/
They write like a log were you start at the beginning of the disk and just walk down the drive never overwriting old data or zeroing anything out.
You get undelete features, the ability to mount a snapshot of the file system at any time in it's history while the real volume is still online, access a file at any time during it's history. That sort of thing. Also has other advantages like very fast write speeds and robustness against loosing data.. even in a file system corruption. (if stuff gets added to the end of a file, just rollback the changes till you get to good data)
Of course it's got problems.. intense file system fragmentation and difficulty with figuring out the best way to reclaim and reuse disk space. It wouldn't be good for general purpose stuff.
snapshots? Posted Jan 28, 2006 14:17 UTC (Sat) by bronson (subscriber, #4806) [Link] This actually sounds darned useful for /home. Right now I have tons of files scattered all over the place that I'm afraid to delete because, who knows, I might need one or two again in the future. With this filesystem I could just go back in time and fetch a file in the rare event that I actually need it again. I'd keep my home dir a lot cleaner.
When the disk is full, it could be put in a maintenance mode where everything is copied as low as possible. This blows away your history, of course, but it clears up fragmentation and recovers unused space.
So... does it work in practice?
snapshots? Posted Jan 30, 2006 2:55 UTC (Mon) by nix (subscriber, #2304) [Link] Well, for what it's worth time-based expiry was designed into Recant from the start. Yes, the algorithm is rather fiddly and expensive; definitely a job to be done by a background thread in times of disk idleness only.
Log-structured filesystems are one of those things that seem terribly neat at the start --- Recant was originally going to be a log-structured FS --- but I spent some time trying to figure out a way to expire them without doing a massive pass over the entire disk and vast memory consumption and never thought of a way. Hence I'm trying something implemented completely differently.
You also can't go back in time on any scale smaller than the entire filesystem with a log-structured FS, which makes it all of marginal use. Recant lets you go backwards on a file-by-file and tree-by-tree basis (with obvious oddities if you have some files in that tree hardlinked to places outside that tree).
However log-structured filesystems are very *efficient* at both reading and writing, fragmentation excepted, and require essentially no maintenance --- until they fill up. But when they fill up, you're in real trouble.
(Now if my hardware would just stop failing I might be able to get some more work done on it. One dead motherboard, one dead network card and one dead disk this weekend alone. *sigh*)
--- oh, and doing complete backups of any filesystem with historical state is a bit of a sod, too. I have some ideas on that point, and oddly after this weekend's disk failures the backup stuff has suddenly started mattering a lot more to me...
snapshots? Posted Feb 2, 2006 16:06 UTC (Thu) by anton (guest, #25547) [Link] Log-structured filesystems are one of those things that seem terribly neat at the start --- Recant was originally going to be a log-structured FS --- but I spent some time trying to figure out a way to expire them without doing a massive pass over the entire disk and vast memory consumption and never thought of a way. Well, I also failed to see a good way for combining the segments and garbage collection ideas of the original LFS proposals with snapshots and clones; moreover, the speed disadvantages of using a free-blocks approach seem to have been mostly eliminated by clustering and delayed writing. But other ideas and properties of log-structured file systems still seem to be worthwhile to me, in particular the possibility of having decent data consistency guarantees and snapshots. So my thoughts have turned to implementing LFSs with mostly conventional free-blocks management, and I have written up these ideas.
snapshots? Posted Apr 12, 2006 19:23 UTC (Wed) by treed (subscriber, #11432) [Link] Sounds like ZODB. ZODB is not a general purpose Unix filesystem but it has many of the qualities you mention.
|
|||||||||||||