Improvements in the block layer [LWN.net]

Improvements in the block layer

Posted Oct 6, 2017 15:20 UTC (Fri) by willy (subscriber, #9762)
Parent article: Improvements in the block layer

> [The NVM Express] driver was incorporated into the mainline kernel in 2012, first appearing in 3.3. It allowed new, fast SSD devices to be run at speed, but that gave no improvement if the block subsystem continued to treat them as pedestrian hard drives. So a new, scalable block layer known as blk-mq (for block-multiqueue) was developed to take better advantage of these fast devices; it was merged for 3.13 in 2014.

I don't know if Jens is spreading false information or if this was a small mistake in transcription, but I heard this inaccurate history given in a talk at LinuxCon NA as well, so I'd like to squelch it. The version of the nvme driver I released absolutely ran at full speed. We got over a million IOPS out of it running against a simulated device. It did this by completely avoiding the queueing (request) layer and taking BIOs at the top of the block layer. Jens wanted to support queueing, so he developed blk-mq. Given the two-year delay in getting blk-mq merged, I'm certain that developing the nvme driver as a bio-based driver was the right decision. I'm still not sure that the request based version of the nvme driver is actually an improvement. It's certainly more complicated!

to post comments

Improvements in the block layer

Posted Oct 8, 2017 5:51 UTC (Sun) by madhatter (subscriber, #4665) [Link]

It was not something Jens said at all, and therefore it was also not a mistake in transcription: it was me adding context from background research. Talks at Linux Recipes were very short on context, because the audience was assumed to be up to speed on all kernel things; that makes for very efficient talks, but without some background they can be a bit difficult for a more general audience to approach. So any such error is entirely mine, and I thank you for the clarification.

Improvements in the block layer

Posted Oct 9, 2017 16:35 UTC (Mon) by axboe (subscriber, #904) [Link]

I've never claimed that the blk-mq version of the nvme driver was any faster, that's not what that conversion was about. It was about reducing the complexity and features that fast drivers had to implement on their own, implementing all of those in a scalable and efficient manner. The goal for the nvme conversion back then was just performance parity. Some things were faster in blk-mq, like the tag generation, and some things obviously made us take a small hit.

> I'm still not sure that the request based version of the nvme driver is actually an improvement. It's certainly more complicated!

IMHO, that statement is obviously false. If you look at the initial conversion, it removed far more lines than it added. It's utilizing shared code in blk-mq that other drivers get to use as well. These days the driver is obviously more complex than the initial version, but that's due to all of the new features that have been added. Years later at this point, I'd say that was definitely the right decision and an improvement. The simpler we can make drivers, the better off we are. Core code is much more heavily scrutinized than driver code, and shared code means we only have to fix bugs once.