Ext3 and RAID: silent data killers?

Posted Aug 31, 2009 22:34 UTC (Mon) by me@jasonclinton.com (subscriber, #52701)
In reply to: Ext3 and RAID: silent data killers? by proski
Parent article: Ext3 and RAID: silent data killers?

SD, MemoryStick, and CF cards do not have firmware.

Ext3 and RAID: silent data killers?

Posted Aug 31, 2009 22:45 UTC (Mon) by pizza (subscriber, #46) [Link] (10 responses)

of course they have firmware; how else would (for example) a CF card translate the ATA commands into individual read/write ops on the appropriate flash chips and deal with write levelling?

Granted, that "firmware" may be in the form fo mask ROM, but I know of at least one case where a CF card had a firmware update released for it.

SD and MS are a lot simpler, but even they require something to translate the SD/MS wire protocols into flash read/write ops.

Ext3 and RAID: silent data killers?

Posted Aug 31, 2009 22:52 UTC (Mon) by me@jasonclinton.com (subscriber, #52701) [Link] (9 responses)

Sorry, you're right about CF. I haven't seen one of those in ages.

Ext3 and RAID: silent data killers?

Posted Aug 31, 2009 23:27 UTC (Mon) by drag (guest, #31333) [Link] (8 responses)

AND SD and Memorystick and any other remotely consumer-related device.

They all are 'smart devices'.

If it was not for the firmware MTD-to-Block translation then you could not use them in Windows and they could not be formatted Fat32.

When I have dealt with Flash in the past, the raw flash type, the flash just appears as a memory region. Like I have this old i386 board I am dealing with that has it's flash just starting at 0x80000 and it goes on for about eight megs or so.

That's it. That's all the hardware does for you. You have to know then how to communicate with it and it's underlining structure and know the proper way to write to it and everything. All that has to be done in software.

I suppose most of that is rather old fashioned.. the flash was soldiered directly into the traces on the board.

I can imagine it would be quite difficult and would require new hardware protocols to allow a OS to manage flash directly properly over something like SATA or USB.

But fundamentally MTD are quite a bit different from Block devices. It's a different class of I/O completely. Just like how a character device like a mouse or a keyboard can't be written to with Fat32. You can fake MTD by running a Block-to-MTD layer on SD flash or a file or anything else and some poeple think that helps with wear leveling, but I think that is foolish and may actually end up being self-defeating as you have no idea how the algorithms in the firmware work.

Ext3 and RAID: silent data killers?

Posted Aug 31, 2009 23:59 UTC (Mon) by BenHutchings (subscriber, #37955) [Link]

AND SD and Memorystick and any other remotely consumer-related device. They all are 'smart devices'.

Not all. SmartMedia, xD and Memory Stick variants provide a raw flash interface - that's a major reason why they have had to be revised repeatedly to allow for higher-capacity chips. They rely on an external controller to do write-buffering, and do not support any wear-leveling layer.

When I have dealt with Flash in the past, the raw flash type, the flash just appears as a memory region

It is possible for a flash controller to map NOR flash into memory since it is random-access for reading. However, large flash chips are all NAND flash which only supports block reads.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 0:00 UTC (Tue) by me@jasonclinton.com (subscriber, #52701) [Link] (6 responses)

Isn't the ATA/MMC<->MTD translation done in the consumer "reader" that you stick these devices in? CF is electrically compatible with ATA. That's not even remotely the case with the electrical interfaces on either SD or MS.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 0:52 UTC (Tue) by drag (guest, #31333) [Link] (3 responses)

Maybe. I don't think so. At least not for SD.

Remember that SD stands for 'Secure Digital' and is DRM'd. So there has to be some smarts in it to do that.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 6:22 UTC (Tue) by Los__D (guest, #15263) [Link] (2 responses)

Almost no SD support the DRM features, according to Wikipedia.

(Still doesn't change the point, though. SDs are probably designed with internal firmware)

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 9:36 UTC (Tue) by alonz (subscriber, #815) [Link] (1 responses)

That's not what Wikipedia says—they say few devices support CPRM. Which is more-or-less true—almost no devices in the western market use CPRM, while in Japan every single device does (it is required as part of i-Mode, which is mandated by DoCoMo).

As for firmware, the SD card interface (available for free at www.sdcard.org defines accesses in terms of 512-byte “logical” sectors, practically mandating the card to implement a flash translation layer.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 12:49 UTC (Tue) by Los__D (guest, #15263) [Link]

Doh, of course.

I read "devices" as the SD cards themselves.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 16:57 UTC (Tue) by Baylink (guest, #755) [Link]

Generally, I think that's true, yes; the only small-flash technology that actually *looks like an ATA drive at the connector* is CF; the others require a smart reader to do the interfacing -- which may itself *not* look like ATA at the back; there are clearly other better ways to do this stuff.

Ext3 and RAID: silent data killers?

Posted Sep 1, 2009 17:37 UTC (Tue) by iabervon (subscriber, #722) [Link]

SD is MMC with a few extra features nobody uses. The readers do USB-storage<->SD, but SD is still 512-byte chunks (it's a card-reported value, and the host can actually try changing it, but 512 is the only value that is ever supported).