Not even close to saturating consumer NVMe disks

Posted Jun 7, 2024 12:03 UTC (Fri) by malmedal (subscriber, #56172)
In reply to: Not even close to saturating consumer NVMe disks by intgr
Parent article: Measuring and improving buffered I/O

Did you try to tune any of the parameters in /sys/block/<device>? I used to have to increase nr_requests quite a bit om large raid-systems.

Not even close to saturating consumer NVMe disks

Posted Jun 7, 2024 14:33 UTC (Fri) by intgr (subscriber, #39733) [Link] (1 responses)

Thanks for the pointer. I tried tweaking various settings in /sys/block/nvme1n1/queue: nr_requests, nomerges, scheduler, write_cache. None of them had positive impact, some decreased performance slightly.

nr_requests was already at 1023 by default and could not be increased. Reducing it decreased performance.

io_poll requires additional boot parameter (nvme.poll_queues), I will try it out later.

Did I miss any important tunables?

Not even close to saturating consumer NVMe disks

Posted Jun 7, 2024 15:40 UTC (Fri) by malmedal (subscriber, #56172) [Link]

No, that's it. In my experience nr_requests as high as it can go and scheduler to none are reliably the best on SSD. In some cases setting cache to write through have been helpful, but that's rare, you almost always want it as write back.

Things keep changing, so it's good to try the options every now and then.