| |||||||||||||
FLASH drives do need a scheduler policyFLASH drives do need a scheduler policyPosted Sep 29, 2004 18:12 UTC (Wed) by BrucePerens (guest, #2510)In reply to: It's not only RAM disks by smurf Parent article: Modular, switchable I/O schedulers FLASH drives like USB sticks have a limited write lifetime. They really should have a scheduler policy, that policty should be to keep frequently-written structures (superblock, FS metadata, directories) in core, consolidate their writes, and write them out infrequently. This is an area where we can come into conflict with the filesystem, which may have its own idea of what should be written atomicaly. BSD-style IO serialization (defining the order in which some critical data must be written, something like tagged queueing but for filesystems) might work best with this scheme, as it would communicate to the scheduler more information about what data it can freely re-arrange and what needs to have its order respected. Thanks Bruce
(Log in to post comments)
FLASH drives do need a scheduler policy Posted Sep 29, 2004 18:17 UTC (Wed) by corbet (editor, #1) [Link] Ah, but decisions like "don't sync the superblock and metadata too often" are not block-level issues, and thus have nothing to do with the I/O scheduler. All filesystems must make their own decisions on where to put data and when to force it to disk - that's higher level stuff. The I/O schedulers discussed in this article, I believe, don't have a role to play there.
FLASH drives do need a scheduler policy Posted Sep 29, 2004 18:40 UTC (Wed) by smurf (subscriber, #17840) [Link] The scheduler can of course notice that a block has been written three times in the last second, and hold further writes for some time.
The downside is that this is excessively unsafe WRT data integrity.
If that's a problem, use a better filesystem than FAT (MP3 sticks, digital cameras, ad nauseam).
FLASH drives do need a scheduler policy Posted Sep 30, 2004 7:45 UTC (Thu) by axboe (subscriber, #904) [Link] No, that's absolutely the wrong layer to attempt to solve that problem, Jon is absolutely right. You should use a suitable file system for such media that has awareness of its limitations.
FLASH drives do need a scheduler policy Posted Oct 1, 2004 0:45 UTC (Fri) by dmaxwell (guest, #14010) [Link] Technically that is true. However, the "killer app" for memory sticks is physically moving data from one system to another. Oftentimes, the systems are running different platforms. My stick can go from a Mac running OS 9 to Windows 2000 to Linux all in one day. My stick would be useless if I used a filesystem "aware of flash memory limitations". The BSDs and Linux offer an embarrassment of riches in filesystems. We can tailor filesystems to the job at hand; imagine that! If you want to move media between platforms then only one filesystem suffices regardless of it's (many) flaws.
I have to use FAT32 if I want to employ my stick as a universal device.
FLASH drives do need a scheduler policy Posted Oct 1, 2004 6:29 UTC (Fri) by axboe (subscriber, #904) [Link] It still doesn't change the fact that you cannot solve this problem at the block layer, as you don't have enough information to do so - all you get is a start offset and length of where to write the data. If you get writes for the same blocks every few seconds, you must look elsewhere to fix it.
And FAT32 is fine to use on a flash stick.
FLASH drives do need a scheduler policy Posted Oct 1, 2004 16:51 UTC (Fri) by BrucePerens (guest, #2510) [Link] Jens,Yes, that's true. However, it can be fixed, if the filesystem communicates either a write barrier (write this block before any blocks I send down after it) or ordering information (write block X before block Y, order of block Z doesn't matter). Once it is so easy to work on I/O scheduling, the need for this will become so evident that there is no question that it will be done. Bruce
FLASH drives do need a scheduler policy Posted Oct 3, 2004 9:48 UTC (Sun) by axboe (subscriber, #904) [Link] Bruce,
We already have write barriers, and it doesn't solve the entire problem. It will make any given fs more flash friendly indeed, but it's still quite a bit away from a fs specifically designed to minimize drive wear.
Your last sentence doesn't make sense.
Jens
FLASH drives do need a scheduler policy Posted Oct 3, 2004 12:06 UTC (Sun) by BrucePerens (guest, #2510) [Link] Jens,OK. I think one of the BSDs goes a bit further than write barriers in communicating time information. But I see your point that this can not go all of the way in reducing FLASH wear. Thanks Bruce
FLASH drives do need a scheduler policy Posted Oct 1, 2004 1:50 UTC (Fri) by BrucePerens (guest, #2510) [Link] Ah, but decisions like "don't sync the superblock and metadata too often" are not block-level issues, and thus have nothing to do with the I/O scheduler.Jon, I understand that to the I/O scheduler, a block is just a block. But I feel that the filesystem should be a higher layer than whatever understands the time constraints of the media. What is necessary is for the filesystem to communicate to the lower levels when order is important. If the I/O scheduler knows that there are 100 blocks that should not go out to the stick without changing the superblock and the directory at the same time, it can handle I/O buffering for USB sticks sensibly. This is why now that we are getting versatile I/O scheduling, some sort of tagged-queueing-like scheme will now become important in the filesystem layer. Thanks Bruce
|
|||||||||||||