An update on bcachefs [LWN.net]

An update on bcachefs

Posted May 24, 2018 6:14 UTC (Thu) by Cyberax (✭ supporter ✭, #52523)
In reply to: An update on bcachefs by rahvin
Parent article: An update on bcachefs

A show-stopping bug in BTRFS RAIDs was discovered only _last_ _year_, after BTRFS had been marked as production-ready for at least a couple of years.

From my experience, I'm using BTRFS on my personal external RAID tower. It got corrupted every single time I tried its multi-device support. Mostly because of botched RAID rebuilds after power failures or disk replacements.

I've got tired of this and switched to a good old MD-based RAID. I'm still using BTFS for snapshots, but there's no way in hell I'm going to touch its multi-device snapshot within the next 5 years or so.

to post comments

An update on bcachefs

Posted May 24, 2018 7:48 UTC (Thu) by khim (subscriber, #9252) [Link] (5 responses)

Have BTRFS developers ever claimed that it's RAID code is production-ready? AFAIK it was always maked red (== experimental, don't use yet) in their table.

An update on bcachefs

Posted May 24, 2018 7:53 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

This page cheerfully says that it's OK: https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_... Back then this page also looked nice and cheerful: https://btrfs.wiki.kernel.org/index.php?title=RAID56&...

An update on bcachefs

Posted May 25, 2018 19:26 UTC (Fri) by khim (subscriber, #9252) [Link] (3 responses)

Ok, I'll bite. You say that "back then" page looked nice and cheerful. Here's the full history.

When exactly RAID56 wasn't marked with red and word "Unstable"?

When you are ignoring warnings written in red... bad things happen... that's just how life is it...

An update on bcachefs

Posted May 25, 2018 20:54 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

"Write hole still exists" is a far cry from "will corrupt your data irrevocably on rebuild".

Also, I wasn't aware of this page's existence. RAID56 search on Google gives another page that looked OK.

An update on bcachefs

Posted May 31, 2018 21:45 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

To be fair, fixing the write hole is tricky - in fact it seems to me that it is inherent and unavoidable in a simple raid 5 implementation.

Even md raid has only just fixed it, with the addition of a journal (a feature that might still be experimental). The journal is intended primarily as an optimisation, flushing updates to SSD before saving them to the full raid on spinning rust. It fixes the write hole almost as a side-effect.

(Note, understanding the journal is on my to-do list, I need to learn more about it before I can document it ... :-)

Cheers,
Wol

An update on bcachefs

Posted Jun 2, 2018 17:52 UTC (Sat) by koverstreet (subscriber, #4296) [Link]

Except btrfs is copy on write - the raid5 hole exists because of doing updates in place on existing stripes, so why btrfs is doing it that way I have no idea.

An update on bcachefs

Posted May 24, 2018 21:50 UTC (Thu) by rahvin (guest, #16953) [Link] (1 responses)

I know it's had issues but it's this bad rap history that follows it around that I think does more reputation damage than the current code deserves. This is one of the reasons it bothers me so much that Redhat decided to devote resources to expanding feature to XFS instead of spending those resources stabilizing btrfs. Btrfs could be IMO the best filesystem Linux has ever had if we could get experienced filesystem developers working on it instead of everyone developing their own filesystem. Which is probably why Redhat's decision bothers me.

It just seems like everyone wants to work on something new rather than try to fix what we've got. I get that, fixing bugs is hard and boring rather than developing something new and shiny. But that new and shiny is still going to take 10 years to stabilize just like every filesystem Linux has had.

This is actually a question I'd like to see answered at one of these filesystem conventions, Why is it so hard to get experienced developers to work stabilizing the fs's we have rather than building new ones?

An update on bcachefs

Posted May 24, 2018 22:51 UTC (Thu) by nivedita76 (subscriber, #121790) [Link]

XFS is around 25 years old, and has been in Linux since the early 00's, not new and shiny. BTRFS is the "new and shiny" fs by comparison. XFS history