| |||||||||||||
snapshots?snapshots?Posted Jan 27, 2006 0:16 UTC (Fri) by drag (subscriber, #31333)In reply to: snapshots? by nix Parent article: MD / DM
There is those "log-structure" filesystems. There are a couple currently in the works for Linux.. one from a telecom company from Japan and another one that made it into that 'google summer of code'.
http://logfs.sourceforge.net/
They write like a log were you start at the beginning of the disk and just walk down the drive never overwriting old data or zeroing anything out.
You get undelete features, the ability to mount a snapshot of the file system at any time in it's history while the real volume is still online, access a file at any time during it's history. That sort of thing. Also has other advantages like very fast write speeds and robustness against loosing data.. even in a file system corruption. (if stuff gets added to the end of a file, just rollback the changes till you get to good data)
Of course it's got problems.. intense file system fragmentation and difficulty with figuring out the best way to reclaim and reuse disk space. It wouldn't be good for general purpose stuff.
(Log in to post comments)
snapshots? Posted Jan 28, 2006 14:17 UTC (Sat) by bronson (subscriber, #4806) [Link] This actually sounds darned useful for /home. Right now I have tons of files scattered all over the place that I'm afraid to delete because, who knows, I might need one or two again in the future. With this filesystem I could just go back in time and fetch a file in the rare event that I actually need it again. I'd keep my home dir a lot cleaner.
When the disk is full, it could be put in a maintenance mode where everything is copied as low as possible. This blows away your history, of course, but it clears up fragmentation and recovers unused space.
So... does it work in practice?
snapshots? Posted Jan 30, 2006 2:55 UTC (Mon) by nix (subscriber, #2304) [Link] Well, for what it's worth time-based expiry was designed into Recant from the start. Yes, the algorithm is rather fiddly and expensive; definitely a job to be done by a background thread in times of disk idleness only.
Log-structured filesystems are one of those things that seem terribly neat at the start --- Recant was originally going to be a log-structured FS --- but I spent some time trying to figure out a way to expire them without doing a massive pass over the entire disk and vast memory consumption and never thought of a way. Hence I'm trying something implemented completely differently.
You also can't go back in time on any scale smaller than the entire filesystem with a log-structured FS, which makes it all of marginal use. Recant lets you go backwards on a file-by-file and tree-by-tree basis (with obvious oddities if you have some files in that tree hardlinked to places outside that tree).
However log-structured filesystems are very *efficient* at both reading and writing, fragmentation excepted, and require essentially no maintenance --- until they fill up. But when they fill up, you're in real trouble.
(Now if my hardware would just stop failing I might be able to get some more work done on it. One dead motherboard, one dead network card and one dead disk this weekend alone. *sigh*)
--- oh, and doing complete backups of any filesystem with historical state is a bit of a sod, too. I have some ideas on that point, and oddly after this weekend's disk failures the backup stuff has suddenly started mattering a lot more to me...
snapshots? Posted Feb 2, 2006 16:06 UTC (Thu) by anton (guest, #25547) [Link] Log-structured filesystems are one of those things that seem terribly neat at the start --- Recant was originally going to be a log-structured FS --- but I spent some time trying to figure out a way to expire them without doing a massive pass over the entire disk and vast memory consumption and never thought of a way. Well, I also failed to see a good way for combining the segments and garbage collection ideas of the original LFS proposals with snapshots and clones; moreover, the speed disadvantages of using a free-blocks approach seem to have been mostly eliminated by clustering and delayed writing. But other ideas and properties of log-structured file systems still seem to be worthwhile to me, in particular the possibility of having decent data consistency guarantees and snapshots. So my thoughts have turned to implementing LFSs with mostly conventional free-blocks management, and I have written up these ideas.
snapshots? Posted Apr 12, 2006 19:23 UTC (Wed) by treed (subscriber, #11432) [Link] Sounds like ZODB. ZODB is not a general purpose Unix filesystem but it has many of the qualities you mention.
|
|||||||||||||