| |||||||||||||
Btrfs: broken by design?Btrfs: broken by design?Posted Jun 24, 2010 12:52 UTC (Thu) by johnflux (guest, #58833)Parent article: Btrfs: broken by design?
I was expecting this article to contain a comment or rebuttal for the quote of the week:
> If you decide to base your file system on some algorithms then please use the original ones from proper academic papers. DO NOT modify the algorithms in solitude: this is very fragile thing! All such modifications must be reviewed by specialists in the theory of algorithms. Such review can be done in various scientific magazines of proper level.
Anyone have any further input on this?
(Log in to post comments)
Btrfs: broken by design? Posted Jun 24, 2010 13:12 UTC (Thu) by NRArnot (subscriber, #3033) [Link]
I put a comment against the QOTW. Basically, it's whether one wants provable behaviour in all circumstances, or whether pragmatism rules. There are very many things that "work", even though you can't prove that they always will. Indeed, there are many useful things which one can prove have a small (hopefully thermodynamically small) chance of total failure.
I can't prove that my head won't fall off before I finish typing this. It's thermodynamically possible, but so unlikely that it won't happen even given a googleplex of universes to try it out i
Btrfs: broken by design? Posted Jun 25, 2010 14:09 UTC (Fri) by tialaramex (subscriber, #21167) [Link]
A really easy example is the "Two generals problem". We know (mathematical proof, and frankly you can work it out with some thought and no training) that you can't achieve the reliable agreement required in this problem. But it appears that many every day things require such agreement. You can't create a TCP connection without it, for example, or buy something with a credit card.
In reality what happens is that we put up with a negligible chance of failure. The failure of this sort are vastly outweighed by more pragmatic problems (imagine if the generals managed their agreement, and then one finds his men have developed dysentery and cannot fight) so we don't worry about it very much.
Btrfs: broken by design? Posted Jun 24, 2010 13:31 UTC (Thu) by masoncl (subscriber, #47138) [Link]
I replied to the thread via patch instead ;) Everyone has different ideas on what a filesystem should be, and my goal isn't to convince everyone that Btrfs fits their definition of ideal.
Ohad Rodeh's paper was really about snapshot reference counting. He did go into details of btree balancing but this was mostly about how top-down balancing is best suited to the snapshot reference counting problem.
We've definitely changed some of the fundamentals around snapshot reference counting too, and we've continued to discuss these with Dr Rodeh (he's really good to work with). His work was presented as a starting point for people interested in snapshotting.
There are also a lot of ideas from reiserfsv3 and xfs and ext in Btrfs. To me, filesystems are really just a collection of tradeoffs and optimizations for specific goals. We trade CPU for disk IO, or CPU and disk IO for features, and these tradeoffs change as we go.
When designing btrfs we took a big pile of known solutions for filesystem problems, crammed them together, and then fixed up the new problems that resulted. Somehow I missed the scientific magazine step that all the other filesystems have followed, but I'll do what I can to keep improving the FS.
Btrfs: broken by design? Posted Jun 24, 2010 16:40 UTC (Thu) by bronson (subscriber, #4806) [Link]
> Somehow I missed the scientific magazine step that all the other filesystems have followed
Can a quote of the week appear in the previous week's comments?
Btrfs: broken by design? Posted Jun 26, 2010 13:52 UTC (Sat) by johnflux (guest, #58833) [Link]
> When designing btrfs we took a big pile of known solutions for filesystem problems, crammed them together, and then fixed up the new problems that resulted. Somehow I missed the scientific magazine step that all the other filesystems have followed, but I'll do what I can to keep improving the FS.
Out of interest, what are your reasons for not having a peer reviewed article on this? Lack of familiarity of the process and/or time, would be my guess?
Especially since you benefited from reading a published paper, it would be nice for you to write up your changes. If you just don't want to go through that bother, maybe a "comment" would be nice? These are half-a-page comments to existing papers but they also get peer reviewed and published.
Actually, you mentioned that you worked with author of the original paper. Maybe he would be interested in publishing your work on your behalf.
Btrfs: broken by design? Posted Jun 29, 2010 20:40 UTC (Tue) by masoncl (subscriber, #47138) [Link]
>Out of interest, what are your reasons for not having a peer reviewed article on this? Lack of familiarity of the process and/or time, would be my guess?
Partially...the part in question is the btree indexing. This is important because it is a core part of the filesystem but it is also the least exotic part of the FS.
The part that is much more performance critical is how do we use that index to track block reference counts and maintain snapshot integrity. Much of my initial work here was expanded by Yan Zheng, and I think his improvements are a better topic for research papers.
Here is a paper where they tried to replace the btrfs back reference tracking and measured some differences:
http://www.usenix.org/events/fast10/tech/full_papers/mack...
Another key part of btrfs is how do we spread checksumming or compression across the CPUs and still maintain good IO ordering.
Or, how do we track free space and not explode (both Josef Bacik and Yan Zheng have spent considerable time on this part).
If I were going to invest time in writing the papers, I'd pick one of those three because they are newer problems where more active research is going on.
|
|||||||||||||