I/O scheduler performance [LWN.net]

I/O scheduler performance

Posted Aug 23, 2010 15:30 UTC (Mon) by epa (subscriber, #39769)
In reply to: A systemd status update by mezcalero
Parent article: A systemd status update

one thing we learned is that maximizing parallelization the way we do has little benefit on the disk elevator on rotating media. Naive people like me assumed that providing the current Linux IO scheduler with a larger amount of requests at the same time it can choose from would improve its performance. Turns out it currently doesn't really.

This is backed up by folk wisdom about how to get fast I/O on Unix-like systems. Do we run multiple cp commands in parallel to give the I/O scheduler more choice? When browsing a directory of image files, does the thumbnail viewer try to open them all simultaneously? We all know that this will tend to make things slower, not faster.

So has research in disk elevator algorithms reached the point where it's possible to do better - to throw large numbers of requests at the system and have it respond to them faster than it would if given one at a time? Or are we stuck (on rotating media at least) with the practical reality that most of the time, requests to the same disk are best made one at a time?

(Log in to post comments)

I/O scheduler performance

Posted Aug 23, 2010 16:31 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]

the problem is that the system has no way of knowing when you submit all this I/O if you mean

do all of these, and minimize the overall time

I need these to all make progress at the same time, even if it means taking longer overall.

current algorithms tend to assume the second, they try to split the available I/O bandwidth between all the requests, since this ends up resulting in lots of seeks, this hurts on traditional media with massive parallel requests

a small amount of parallelism helps by giving the drive something to do when it would otherwise be idle, however once you pass the saturation point it hurts because it adds additional seeks as the system jumps from one set of requests to the next.

this is the same sort of thing that makes hyperthreading be anywhere from a noticable benifit to a mild loss depending on the workload

I/O scheduler performance

Posted Aug 23, 2010 20:04 UTC (Mon) by axboe (subscriber, #904) [Link]

A scheduler like CFQ will attempt to provide a mix of what you describe, depending on how you submit it. If the submission is done from one process, it will assume that you want it to be done as fast as possible. It'll be sorted accordingly. If done from multiple processes or threads, it will attempt to provide equal progress while preserving overall throughput.

What you describe is true on classical work conserving IO schedulers, it's not the case for the default Linux IO scheduler.

I/O scheduler performance

Posted Sep 8, 2010 11:15 UTC (Wed) by epa (subscriber, #39769) [Link]

So, then, the way to get fast I/O is to make asynchronous I/O calls from a single thread (so that the scheduler knows that fairness doesn't matter) rather than spawning multiple threads or processes.

Is there any way to fork subprocesses but still let CFQ know that they're all related and happy to altruistically share I/O bandwidth between them, so it doesn't try to slice up I/O requests fairly at the expense of total throughput?