Defending against page-cache attacks

The page cache allows both write aggregation and readahead, and for writes to complete asynchronously from the submitting syscall. Both of these have enormous (positive) performance impacts which rise as the amount of I/O the filesystem/device can have in flight increases, and also as the response latency of the device increases.

The page cache allows your single threaded dd to have the system queue up a bunch of writes which may be able to be processed all at once, as contrasted with direct I/O which is 1 I/O per process.

Additionally, if your whole write fits in the page cache and you’re not doing other heavy I/O (ie semi-idle time is available to write out your data) the ability to write to memory and complete asynchronously means your application level performance (where the app doesn’t wait for the write to be on disk) will stomp almost any standard storage device or RAID array,

This means it’s not beneficial to use direct I/O for single use I/O in general, it really depends on your case. DIO is essentially only faster in the cases where your device is *extremely* fast or you have many threads and a very high bandwidth back end (you can overwhelm the page cache).

In cases with higher latency devices (HDD, network file systems) or where there is device level parallelism to exploit (SSDs), direct I/O is often much, much slower, even for well formed I/O. (In real deployments of the Lustre parallel file system, which I work on, single threaded DIO can be 5-10x slower than normal I/O. That’s an extreme case but the reasons for it hold for local file systems too.)