if the queue size is not large enough, then you can't fit enough requests into the queue to have them available to combine later.
If the queue size is too large, then a new process making a request will not get it's request serviced until everything ahead of it in the queue gets processed (unless you have some fairness process to not put the new processes request at the end of the queue)
I don't like the concept of plugging, but it seems to be a hack that tends to work.
as an example.
In rsyslog, when the ability to process multiple messages from the queue at once was added (so that multiple messages could be inserted to a database in a single transaction for example), we discussed delaying pulling the first message from the queue to give the queue a chance to build up several messages that would then be handled more efficiently (in one pass), but we decided to not do this because the process ended up being self-regulating.
If the messages arrived slowly enough, they are handled one at a time.
If the messages arrive faster than this, some messages queue up while the prior messages are handled and then the backlog gets processed at one time (up to a limit)
This is very good for latency, but the trade-off is that the output is doing far more work than it would need to do if the work was batched more. As the load builds up, it will ramp up the utilisation of the output in the most inefficient mode (one message at a time), and then when it saturates the output, it will become more efficient to process more messages while keeping the output at max utilisation.
Networks have the same type of problem (the too large buffer situation is what's called bufferbloat), the answer there seems to be to put in a more complex queuing engine (SFQ seems to be the winner right now) that priorities packets from new or sparse connections ahead of heavy connections.
I wonder if a similar approach could work for disk I/O? If this would allow for significantly larger queue sizes without the latency problems that usually come with large queues, it may give almost the same long-term effect of plugging, without the problems that plugging introduces.