Stable pages

One kind of workload that's affected negatively would be any low-latency process which writes to disk.
When I do that, in order to guarantee that the main program responds immediately I lock the whole application in memory and use a separate writing thread.
But if you lock a couple of my process' pages when writing, that lock will affect unrelated data structures which simply happen to be on the same meory page. I can thus no longer guarantee that my main task will no longer block on random memory writes. That's not acceptable.

Perhaps I'm misunderstanding, but won't a series of small sequential writes trigger wait-on-writeback? Or does this not apply to appending to a file-backed page?

In a previous life I was involved in the performance measurement of coalescing disk writes, and we found a very large number of sequential writes could be coalesced into single writes, and then adjacent blocks coalesced into larger singe writes. This paid off particularly well when a disk was being handed writes at or beyond it's capacity, by removing unneeded writes. I think I still have the graphs somewhere (;-))

I'll comment on the non-sequential case in a sec, after I look at my archive.

--dave

I had a look at the paper the work I measured was based on, and wonder if we're really looking at a race condition: we take a checksum, queue the data for I/O and compare the data as part of or after the I/O to see if an error has occurred.

Delaying, duplicating or COWing allows us to survive or avoid the data changing while the I/O is queued, which is a pretty long time compared to anything happening in main memory. The speed difference gives us a relatively large period in which a program can race ahead of the disk.

If the purpose is to validate the disk write, one would want to do the checksum as late as possible before the write, and verify it either as part of hardware write or via a read-after-write step. That keeps the time period tiny.

If the purpose is to validate it from end to end, I suspect you need more than one check. One check would need to be done as the data is queued, to be sure it made it to the queue ok, which would need to be amended if the page in queue is coalesced with a later write. In the latter case you have a new, amended checksum to check as-or-after the write.

Alas, I'm not following the main list these days, so I'm unclear of the fine details of the requirements you face!

--dave