| |||||||||||||
Winmodem-like solid state storageWinmodem-like solid state storagePosted Apr 11, 2009 23:09 UTC (Sat) by jzbiciak (✭ supporter ✭, #5246)In reply to: Winmodem-like solid state storage by giraffedata Parent article: Linux Storage and Filesystem Workshop, day 2
So it still looks to me like Winmodem-style flash storage would be a bane to Linux and free software. Nonsense. There are plenty of machines out there that have just plain NAND or NOR flash hooked up to the CPU and Linux reads/writes these effectively. The issue is that currently we only see that in the embedded space, and it's typically just enough flash to hold the code for whatever that machine is supposed to do. For example, be a WiFi router or a set-top box or a cell phone. What I'd like to see is something I can get off the shelf at my local Computer Mart (or on the web) that plugs into my PC and gives me raw flash. Instead of focusing on "right sized" and "small" and "maximizing battery life", it instead can be a bank of parallel flash such as what Intel's SSD disks are, but with a raw interface. We can then use our existing flash filesystems and infrastructure to drive those in a desktop and laptop space, rather than just the netbook/smart-phone/smart-router space. Now, these (potentially) massively parallel performance oriented disks will need additional software support. You want something akin to RAID striping across the media along with maybe some redundancy in addition to wear leveling. That's just enhancements on top of our existing wear leveling filesystems and infrastructure. The only real issue is that once you give raw flash to the OS and put the smarts in the OS, it'll be harder for dual-boot systems to communicate on the same media, because the likelihood that $VENDOR's Windows driver organizes the disk the way Linux does is slim to none unless $VENDOR works with the Linux community also.
(Log in to post comments)
Winmodem-like solid state storage Posted Apr 12, 2009 1:21 UTC (Sun) by dwmw2 (subscriber, #2063) [Link] "The only real issue is that once you give raw flash to the OS and put the smarts in the OS, it'll be harder for dual-boot systems to communicate on the same media, because the likelihood that $VENDOR's Windows driver organizes the disk the way Linux does is slim to none unless $VENDOR works with the Linux community also."I see two reasons why that wouldn't be a problem, in practice. Firstly, we've never had many problems working with "foreign" formats. We cope with NTFS, HFS and various bizarre crappy "Software RAID" formats, amongst other things. That includes the special on-flash formats like the NAND Flash Translation Layer used on the M-Systems DiskOnChip devices, which has been supported for about a decade. Are you suggesting that hardware vendors take Linux less seriously now than they did ten years ago, and that we'd have a harder time working out how to interoperate? Remember, documenting the on-medium format doesn't necessarily give away all the implementation details like algorithms for wear levelling, etc. — that's why M-Systems were content to give us documentation, all that time ago. Secondly, interoperability at that level isn't a showstopper. It's nice to have, admittedly, but I'm not going to lose a lot of sleep if I can't mount my Windows or MacOS file system under Linux. It's the native functionality of the device under Linux that I care about most of the time. Of course, I see no reason why the device vendors should be pushing their own "speshul" formats anyway — the hard drive vendors don't. But I'm not naïve enough to think that they won't try. Imagine a world where every hard drive you buy is actually a more like a NAS. You can only talk a high-level protocol like CIFS or NFS to it; you can't access the sectors directly. Each vendor has their own proprietary file system on it internally, implemented behind closed doors by the same kind of people who write BIOSes. You have no real information about what's inside, and can't make educated decisions on which products to buy. Having made your choice you can't debug it, you can't optimise it for your own use case, you can't try to recover your data when things go wrong, and you sure as hell can't use btrfs on it. All you can do is pray to the deity of your choice, then throw the poxy thing out the window when it loses your data. If the above paragraph leaves you in a cold sweat, it was intended to. That's the kind of dystopia I see in my head, when we talk about SSDs without direct access to the flash.
Winmodem-like solid state storage Posted Apr 12, 2009 1:46 UTC (Sun) by giraffedata (subscriber, #1954) [Link] What I'd like to see is something I can get off the shelf at my local Computer Mart (or on the web) that plugs into my PC and gives me raw flash. If a PCIe expansion socket is sufficient, several companies are now selling that. I remember IBM demonstrating last year a prototype storage server composed of a bunch of Linux systems with Fusion-IO PCI Express cards for storage. It broke some kind of record. In that system, the flash storage still appeared as a block device, but it did it at the Linux block device interface instead of at the SCSI physical interface.
Winmodem-like solid state storage Posted Apr 16, 2009 18:53 UTC (Thu) by wmf (guest, #33791) [Link]
Fusion io is not raw flash since the driver contains a sophisticated FTL that cannot be disabled. In theory they could release an MTD driver, but they're not going to.
Winmodem-like solid state storage Posted Apr 16, 2009 20:15 UTC (Thu) by giraffedata (subscriber, #1954) [Link] Fusion io is not raw flash since the driver contains a sophisticated FTL that cannot be disabled. In theory they could release an MTD driver, but they're not going to. Is the driver you're talking about a Linux kernel module? An object code only one? What is MTD?
All this has happened before... Posted Apr 16, 2009 21:55 UTC (Thu) by wmf (guest, #33791) [Link] Is the driver you're talking about a Linux kernel module? An object code only one?Yep. What is MTD?How you access raw flash. See also UBIFS Raw flash vs. FTL devices.
Winmodem-like solid state storage Posted Apr 19, 2009 14:11 UTC (Sun) by oak (subscriber, #2786) [Link]
> What I'd like to see is something I can get off the shelf at my local
Computer Mart (or on the web) that plugs into my PC and gives me raw flash. Instead of focusing on "right sized" and "small" and "maximizing battery life", it instead can be a bank of parallel flash such as what Intel's SSD disks are, but with a raw interface. We can then use our existing flash filesystems and infrastructure to drive those in a desktop and laptop space
Unlike block based file systems like ext[234], the existing flash file
However, the newly merged UBIFS promises to work much better:
There's not usage data on how well it performs with desktop and server
Winmodem-like solid state storage Posted Apr 19, 2009 14:28 UTC (Sun) by dwmw2 (subscriber, #2063) [Link] " Unlike block based file systems like ext[234], the existing flash file systems are designed for very small file systems. E.g. JFFS2 keeps the whole file system metadata in RAM and is unusable in GB sized file systems."Very true — although we put a lot of effort in to make JFFS2 better for OLPC with its 1GiB of NAND flash. It mounts in 6 seconds or so, and we reduced the RAM usage by a significant amount too. But still, JFFS2 was designed in the days of 32MiB or so of NOR flash, and definitely isn't intended to scale up to the kind of sizes we're seeing now. UBIFS is much more promising, but as you correctly observe is not yet proven for desktop or server workloads. I'm actually keen to get btrfs working on raw flash, too. The point is that with stuff done in software, we can do better; whether we do better or not today is a different, and less interesting issue. After all, we can always implement the same "pretend to be a block device" kind of thing to tide us over in the short term, if we need to. We have three or four such translation layers in Linux already, and more on the way.
|
|||||||||||||