User: Password:
Subscribe / Log in / New account

Explicit block device plugging

Explicit block device plugging

Posted Apr 29, 2011 14:57 UTC (Fri) by dlang (subscriber, #313)
In reply to: Explicit block device plugging by neilbrown
Parent article: Explicit block device plugging

note that in my example, the timer was only used to indicate the max amount of time to wait for the next item to be submitted.

how can the kernel know when the application has finished submitting a bunch of requests?

or is it that the application submits one request, but something in the kernel is breaking it into a bunch of requests that all get submitted at once, and plugging is an attempt to allow the kernel to recombine them? (but that doesn't match your comment about sorting 3 non-adjacent requests being a win, how can one action by an application generate 3 non-adjacent requests?)

I'm obviously missing something here.

if the application is doing multiple read/write commands, I don't see how the kernel can possibly know how soon the next activity will be submitted after each command is run.

if the application is doing something with a single command, it seems like the problem is that it shouldn't be broken up to begin with, so there would be no need to plug to try and combine them

(Log in to post comments)

Explicit block device plugging

Posted Apr 29, 2011 22:58 UTC (Fri) by neilbrown (subscriber, #359) [Link]

The actual times between plug and unplug are typically microseconds (I suspect). The old timeout was set at 3 milliseconds and that was very slow. It is almost nothing compared to device IO times.

Actions of the application and requests to devices are fairly well disconnected thanks to the page cache. An app writes to the page cache and the page cache doesn't even think about writing to the device for typically 30 seconds. Of course if the app calls fsync, that expedites things. So a partial answer to "how can the kernel know when the application has finished submitting a bunch of requests" is "the application calls 'fsync' - if it cares".

On the read side, the page cache performs read-ahead so that hopefully every read request can be served from cache - and certainly the device gets large read requests even if the app is making lots of small read requests.

Also the kernel does break things into a bunch of requests which then need to be sorted. If a file is not contiguous on disk, then you need at least one request each separate chunk. Plugging allows this sorting to happen before the first request is started.

There is a good reason why the page cache submits lots of individual requests rather than a single list with lots of requests. Every request requires an allocation. when memory gets tight (which affects writes more than reads) it could be that I cannot allocate memory for another request until the previous ones have been submitted and completed. So we submit the requests individually, but combine them at a level a little lower down, and 'unplug' that queue either when all have been submitted or when the thread 'schedules' - which it will typically only do if it blocks on a memory allocation.

So there are two distinct things here that could get confused.

Firstly there is the page cache which deliberately delays writes and expedites reads to allow large requests independent of the request size used by the application.

Then there is the fact that the page cache sends smallish requests to the device, but tends to send a lot in quick succession. These need to be combined when possible, but also flushed as soon as there is any sign of any complication. This last is what "plugging" does.

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds