Defending against page-cache attacks [LWN.net]

Defending against page-cache attacks

Posted Jan 18, 2019 14:47 UTC (Fri) by bof (subscriber, #110741)
In reply to: Defending against page-cache attacks by mangix
Parent article: Defending against page-cache attacks

> wonder how many programs will use O_DIRECT now.

Anything with a use case that wants to *avoid* perturbing the page cache. As a sysadmin I regularly use dd iflag=direct or oflag=direct when checksumming or network copying block devices. Applicable to all do-once I/O, actually, and the last time I played with fadvise FADV_NOREUSE (which dd does not support anyway) it was much less reliable.

to post comments

Defending against page-cache attacks

Posted Jan 21, 2019 1:49 UTC (Mon) by Paf (subscriber, #91811) [Link]

The page cache is not just about reuse.

The page cache allows both write aggregation and readahead, and for writes to complete asynchronously from the submitting syscall. Both of these have enormous (positive) performance impacts which rise as the amount of I/O the filesystem/device can have in flight increases, and also as the response latency of the device increases.

The page cache allows your single threaded dd to have the system queue up a bunch of writes which may be able to be processed all at once, as contrasted with direct I/O which is 1 I/O per process.

Additionally, if your whole write fits in the page cache and you’re not doing other heavy I/O (ie semi-idle time is available to write out your data) the ability to write to memory and complete asynchronously means your application level performance (where the app doesn’t wait for the write to be on disk) will stomp almost any standard storage device or RAID array,

This means it’s not beneficial to use direct I/O for single use I/O in general, it really depends on your case. DIO is essentially only faster in the cases where your device is *extremely* fast or you have many threads and a very high bandwidth back end (you can overwhelm the page cache).

In cases with higher latency devices (HDD, network file systems) or where there is device level parallelism to exploit (SSDs), direct I/O is often much, much slower, even for well formed I/O. (In real deployments of the Lustre parallel file system, which I work on, single threaded DIO can be 5-10x slower than normal I/O. That’s an extreme case but the reasons for it hold for local file systems too.)