LWN.net Logo

Linux and object storage devices

Linux and object storage devices

Posted Nov 5, 2008 2:01 UTC (Wed) by gdt (subscriber, #6284)
Parent article: Linux and object storage devices

OSD isn't really aimed at drives, it's aimed at managed storage devices. At the moment these either need to handle disk blocks, which is pretty low level, or implement a network file system, which is pretty high level. You really want a protocol which allows the managed storage to understand which blocks comprise a file with buying into all of the semantics of file handling. That allows the storage to migrate the file to nearline storage, duplicate the entire file for redundancy, etc.

The protocol is an extension to SCSI not because OSD drives are expected to be common, but because that's the transport above Fiber Channel and iSCSI -- the dominant storage attachment protocols.


(Log in to post comments)

Linux and object storage devices

Posted Nov 5, 2008 5:36 UTC (Wed) by flewellyn (subscriber, #5047) [Link]

So it doesn't seem likely to you that OSDs will supercede block devices for "commodity" mass storage? That's good...I don't really like the idea of the hardware itself mandating a particular organizational scheme.

Linux and object storage devices

Posted Nov 8, 2008 22:51 UTC (Sat) by vonbrand (subscriber, #4458) [Link]

So you don't like the layout in blocks of your current disk drive either...

Nice exaggeration

Posted Apr 9, 2009 7:24 UTC (Thu) by felixfix (subscriber, #242) [Link]

I want each byte to include a length specifier so I can have variable size and capability tickets with timeouts per byte. I resent the rigid 8 bit byte paradigm enforced upon me by hardware manufacturers.

The difference between hardware enforced block size and hardware enforced security, object layout, capability tickets, and all the other cruft that goes with it is obvious to most people. If you don't see it, you shouldn't be wasting your time reading news from a Linux web site, which places a premium on cognitive capability.

Linux and object storage devices

Posted Nov 5, 2008 6:50 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]

so why would you use OSD instead of one of the many network/cluster file systems? they all abstract the block device away and let the server at the far side do whatever it wants for the data storage.

the only 'advantage' this would have is that instead of using cheap, common ethernet, it requires a SCSI layer (parallel SCSI or fiberchannel), you could mix and match normal storage with OSD storage, but they really are so different that I would not expect them to be mixed in practice.

Linux and object storage devices

Posted Nov 5, 2008 8:36 UTC (Wed) by jamesh (guest, #1159) [Link]

They probably want something that isn't quite as high level as a NAS box such as abstraction of the actual location of the file data (flash, disk, tape, etc), while not imposing a particular high level file system interface as you'd get with NFS or SMB.

It'd probably also make it easy to do things like run multiple file system hierarchies off the same object store, rather than having to worry about growing or shrinking partitions.

Lastly, this doesn't look incompatible with using ethernet to connect to the storage device: people have been running SCSI over ethernet for years with iSCSI.

Linux and object storage devices

Posted Nov 6, 2008 12:01 UTC (Thu) by csamuel (✭ supporter ✭, #2624) [Link]

This *has* come from the cluster filesystems, I first came across OSD's in 2004 as
part of the Lustre file system design. The idea is that you have a whole bunch of
OSD's and you parallelise your I/O across them.

Lustre is now owned by Sun...

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds