Support for shingled magnetic recording devices

Posted Mar 30, 2014 16:34 UTC (Sun) by giraffedata (guest, #1954)
In reply to: Support for shingled magnetic recording devices by khim
Parent article: Support for shingled magnetic recording devices

And when used with GFS or HDFS which need random read access to it's 64 megabyte chunks and kinda streamlined write access to these chunks (GFS gives you the ability to append-write files, HDFS does not even offer that today) they are fast.

I think those are two examples of things that would require re-engineering to work with shingled disks, because neither writes in log-structured fashion today. The only reason I can think of that an existing disk drive application would write in log-structured fashion (fill the drive, or a large segment of it, from beginning to end) is to maximize write speed. But GFS and HDFS assume there is little writing. HDFS is specifically aimed at fast sequential reading of large files, which means it needs to keep files contiguous on disk, which is not possible with a log structured file system.

Support for shingled magnetic recording devices

Posted Mar 31, 2014 10:36 UTC (Mon) by dlang (guest, #313) [Link]

I think that you wildly underestimate how expensive random writes are today, and how much software works to avoid them

I've seen a number of packages that have the pattern of having large chunks of data, but new data is not written directly to those large chunks, instead new data is written sequentially to an 'updates' file, and periodically some other job comes along and re-writes the large chunks to include the changes from the updates files, and then deletes the updates files.

such systems would be perfect for shingled drives, they would just get their chunk sizes and alignments adjusted to match.