Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 7:49 UTC (Fri) by zuzzurro (subscriber, #61118)
Parent article: Fedora 16 to use Btrfs as its default filesystem

What I really would like to know is whether the concerns voiced by mr. Edward Shishkin (see this LWN article: http://lwn.net/Articles/393144/) were real or not.
I never actually saw him take them back even after the long discussions with the btrfs developers. So I would really like to hear his opinion and I would love to have LWN interview him about this.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 12:23 UTC (Fri) by MisterIO (guest, #36192) [Link]

I agree, I would like to see an article about this or an interview or something like that.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 13:14 UTC (Fri) by ricwheeler (subscriber, #4980) [Link] (9 responses)

Edward pointed out a bug that Chris and others discussed with him and resolved. Not a fundamental design issue or flaw in btrfs.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 13:26 UTC (Fri) by MisterIO (guest, #36192) [Link] (8 responses)

Not really. They found some bugs(based on that review) and solved them, but other fundamental design problems described on that review weren't really solved/aknowleged. I've seen a rebuttal by Mason, stating that many(all?) of the concerns were actually already solved in the code(describing in detail which function does what). I didn't see an answer from Shishkin to that. So either Shishkin was wrong(maybe because he didn't take a deep enough look at the code), or the problems aren't solved.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 15:26 UTC (Fri) by ricwheeler (subscriber, #4980) [Link] (7 responses)

Edward is not actively working on btrfs at the moment which might explain the lack of an update from him (and he and Josef both work in my group at Red Hat).

I would take Chris on his word - has he ever given you reason to doubt his skills at writing file systems :) ?

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 16:12 UTC (Fri) by MisterIO (guest, #36192) [Link] (6 responses)

I'm currently using btrfs on 10+ partitions on my desktop, against 3 with ext4 and one with ntfs, so that should show the level of my concern about this. As I've said somewhere else, the main problem of btrfs, currently, is the lack of a working btrfs. That said, faith is not for engineers. Even the best coders make mistakes. It's just that to me it seems like the explanation about why btrfs doesn't have any serious design problems hasn't really come out in a clear form, compared instead to the explanation of Shishkin doubts.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 17:59 UTC (Fri) by nix (subscriber, #2304) [Link] (5 responses)

the main problem of btrfs, currently, is the lack of a working btrfs.

That would tend to be problematic, yes.

(I think the latter btrfs should perhaps be 'btrfsck'?)

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 18:07 UTC (Fri) by MisterIO (guest, #36192) [Link] (4 responses)

yeah, I meant btrfsck.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 18:21 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (3 responses)

Just in case, you missed it

https://lwn.net/Articles/446984/

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 18:27 UTC (Fri) by MisterIO (guest, #36192) [Link] (2 responses)

No, I didn't miss it. That's really good news, but currently the code is still missing, so the problem is still present.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 18:33 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (1 responses)

It is not "missing". It is being tested upstream and when upstream developers are convinced that it is ready, it will be included.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 18:42 UTC (Fri) by MisterIO (guest, #36192) [Link]

AFAICT it can't be found in any public repository, so it _is_ still missing.

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 19:49 UTC (Fri) by masoncl (subscriber, #47138) [Link] (6 responses)

Edward is a smart guy, and he did find a real bug in the btrfs balancing code (which we fixed).

He also had a lot of comments about the balancing code in general. Basically he didn't feel we were really balancing the btrees and had a long list of things he felt well were missing. I went through it and explained where in the Btrfs code each one was happening.

The part I don't think he ever agreed with is the way Btrfs stores inline file data. In reiser v3 (and I think v4), the inline file data for a single file can be split between btree leaves. This means they are exceptionally space efficient for packed small files.

But, it also leads to complexity in the balancing code, and in the code to read/write the inline file data. In Btrfs I chose to never split the file data that gets packed into btree leaves. It either fits in one item in one leaf, or it gets tossed into an extent outside of the btree.

This leads to more wasted space in the btree leaves, which is doubled because Btrfs duplicates metadata blocks by default (even on single device filesystems).

The percentage of wasted space increases as your inline file size approaches half the size of a single btree leaf (4k right now). If all your files are 2k, you'll end up with a lot of btree leaves 50% full. This is less of a problem if the btree leaf size is larger, which we do plan to support in the 3.1 kernel.

Complex systems are full of compromises, and Btrfs certainly has its fair share.

-chris

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 20:14 UTC (Fri) by MisterIO (guest, #36192) [Link] (5 responses)

yes, I remember that part. the way it is now, the space waste isn't limited, right? that is, it doesn't have an upper bound. wouldn't it be possible to say something like: if the space waste goes over a certain limit(a percentual of the total space), stop inlining file data, go back to using extents?

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 20:34 UTC (Fri) by masoncl (subscriber, #47138) [Link] (4 responses)

We have mount -o max_inline=, so the admin can control the max size that we choose to inline. The default is a little under 4KB, which works well most of the time. Usually we're able to get the inode, xattrs and file data all in the same block, so even for a 2KB file it works well.

The problem comes when they are all 2KB files, which is why we give the admin knobs.

For larger leaf sizes, we'll still have a limit of 4KB inlined in the btree, so the overall percentage will be a lot lower. With a 16KB leaf, 2KB inlined files won't be a problem.

-chris

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 21:14 UTC (Fri) by MisterIO (guest, #36192) [Link] (1 responses)

As I see it, inlining file data is an optimization for small files, right? Big files use extents by default, right? So the obvious solution to the problem would be to say: if the percentual of wasted space is too high, stop optimizing for small files, treat all files as big files and just put file data on extents. Now, being this so obvious, it just means that, being a noob on fs programming, I can't see the obvious problem with that solution(maybe extreme performance degradation? maybe fragmentation? maybe both?), or maybe that it wouldn't be a solution at all.
Anyway, I appreciate that you took the time to explain this here.
It would be useful if larger leaf sizes could be used even on old btrfs filesystems(that is, I hope it's not an option to be specified at fs creation time).

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 23:43 UTC (Fri) by masoncl (subscriber, #47138) [Link]

Part of the answer is just that maintaining an overall "how much space have we wasted number" is yet another metric to store and maintain, and it does tend to get tricky to count in the face of snapshots. So there's a real CPU cost involved.

But the main reason is that I prefer the filesystem be predictable. If your file size is less than a specific value, it goes into the btree. It's very simple to explain and isn't surprising ;)

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 21:36 UTC (Fri) by jengelh (guest, #33263) [Link] (1 responses)

What makes "a little under 4KB" better than xfs's choice of 256 bytes? (Which itself I generally bumped to 512 on fses where a 'significant' amount of files may carry ACLs.)

Fedora 16 to use Btrfs as its default filesystem

Posted Jun 10, 2011 23:49 UTC (Fri) by masoncl (subscriber, #47138) [Link]

Better is a tricky word. 4KB is the smallest extent we'll allocate, so being able to allocate up to 4KB in the btree usually makes for less wasted space.

There are a lot of factors in play. No inline data means more inodes in a block, which means things that stat in bulk (or rm) tend to be faster.

But, inline data means that when we read the inode we have the data, so things like reading all the files in a directory tend to be faster.

-chris