Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 3:16 UTC (Tue) by zlynx (guest, #2285)
In reply to: Ext3 and RAID: silent data killers? by me@jasonclinton.com
Parent article: Ext3 and RAID: silent data killers?

All the RAID discussion on the list was about the Linux MD/DM software RAID. It isn't as reliable as other options.

From what I gather, MD does not use write-intent logging by default, and when it is enabled it is very inefficient. Probably because it doesn't spread the write intent logs around the disks. Also, MD does not detect a unclean shutdown, so it does not start a RAID scrub and go into read+verify mode. And all that is a problem even when the array isn't degraded.

And of course it doesn't have a battery backup. :)

All that said, Linux MD hasn't given me any problems, and I prefer it over most cheap hardware RAID.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 5:14 UTC (Tue) by neilbrown (subscriber, #359) [Link] (1 responses)

I don't think "is very inefficient" is correct. There is a real performance impact, but the size of that impact is very dependent on workload and configuration. It is easy to add or remove the write intent logging while the array is active, so there is minimal cost in experimenting to see what impact it really has in an given usage.

And MD most certainly does detect an unclean shutdown and will validate all parity block on restart.

But you are right that it doesn't have battery backup. If fast NVRAM were available on commodity server hardware, I suspect we would get support for it in md/raid5 in fairly short order.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 15:30 UTC (Tue) by zlynx (guest, #2285) [Link]

As far as I could tell, MD will not verify parity during regular reads while the array is unclean.

It may start a background verify, although it seemed to me that was dependent on what the distro's startup scripts did...