> Networking technology went from 10Mb/s in the 1980's to 10Gb/s now, the better part of 30 years later. SSDs have forced a similar jump (three orders of magnitude) in a much shorter period of time - and every indication suggests that devices with IOPS rates in the millions are not that far away.
Probably the main reason why such an unfortunate IOPS jump has been forced in networking is backward compatibility. Jumbo frames? Fail because of backward compatibility. Evolving TCP/IP to ease hardware assistance? Fail because of backward compatibility. Etc.
That is because the backward compatibility requirement is nowhere as strong as in networking. You can easily upgrade your PC. It is even reasonably easy to upgrade your company-wide software. But good luck trying to upgrade the Internet. Or even just Ethernet. See IPv6 for instance: it comes as a brand new feature practically not touching anything already in place, but even such a smooth "upgrade" is a hard sell!
One of the unfortunate consequences is that transferring a DVD image on the network requires millions of IOPS all across the path.
In comparison, the need for backward compatibility in storage is basically inexistent. So this network/storage analogy must stop somewhere. Please someone from the storage camp tell us where exactly. Surely reading or writing a DVD image to disk does not/will not require millions of IOPS. Or will it still?
Posted Oct 5, 2010 11:13 UTC (Tue) by axboe (subscriber, #904)
[Link]
Sequential IO will of course use larger IO sizes. The IOPS quest is largely for the mainly randomized IO workloads.
Solid-state storage devices and the block layer
Posted Oct 5, 2010 18:21 UTC (Tue) by angdraug (subscriber, #7487)
[Link]
See IPv6 for instance: it comes as a brand new feature practically not touching anything already in place, but even such a smooth "upgrade" is a hard sell!
Have you seen this article at ArsTechnica? It goes to some lengths to explain the problems with IPv6 transition. If it's to be believed, IPv6 transition is quite far from "smooth".
Solid-state storage devices and the block layer
Posted Oct 5, 2010 23:30 UTC (Tue) by marcH (subscriber, #57642)
[Link]
> If it's to be believed, IPv6 transition is quite far from "smooth".
Yes but it would have been much worse (read: impossible) if IPv6 deployment ever required substantial changes to IPv4.
This is an interesting article. Except they are wrong when they pretend it is easy to break backward-compatibility with Ethernet or TCP. It is not easy but only "less impossible" than breaking IPv4 backward compatibility.
Note: the focus of the article is obviously neither on Ethernet nor on TCP.
Solid-state storage devices and the block layer
Posted Oct 8, 2010 23:48 UTC (Fri) by giraffedata (subscriber, #1954)
[Link]
Probably the main reason why such an unfortunate IOPS jump has been forced in networking is backward compatibility.
...
In comparison, the need for backward compatibility in storage is basically inexistent.
Well, the the whole reason SSDs exist is backward compatibility with rotating media, and it does slow things down considerably. If not for backward compatibility, we wouldn't use SCSI or even Linux block devices to access solid state storage. Write amplification by read-modify-write wouldn't be a problem if the device weren't trying to emulate a 512-byte-sectored disk drive.
Existence of SSDs tells me people aren't willing to replace the entire system at once -- they want to replace just the disk drives.
Not knowing the network issues, though, I can believe that backward compatibilty hinders performance less in storage than for ethernet.