|
|
Subscribe / Log in / New account

There's a better solution than falling back to buffered io

There's a better solution than falling back to buffered io

Posted Nov 12, 2025 19:26 UTC (Wed) by koverstreet (✭ supporter ✭, #4296)
In reply to: There's a better solution than falling back to buffered io by nickodell
Parent article: The intersection of unstable pages and direct I/O

Buffered IO is only fast when IO is mostly staying in cache. When it's not it's _significantly_ slower; you incur all the overhead of walking and managing the page cache radix tree, backround eviction, background writeback - doing all that stuff asynchronously is significantly more constantly when you're not amortizing it by letting things stay in cache.

O_DIRECT tends to be used where the application knows the pagecache is not going to be useful - ignoring what the application communicated and using buffered IO is a massive behavioral change and just not a good idea.

If you just bounce IOs, the only extra overhead you're paying for is allocating/freeing bounce buffers (generally quite fast, thanks to percpu freelists), and the memcpy - which, as I mentioned, is noise when we're already touching the data to checksum.

And you only have to pay for that on writes. Reads can be normal zero copy O_DIRECT reads: if you get a checksum error, and the buffer is mapped to userspace (i.e. might have been scribbled over), you retry it with a bounce buffer before treating it like a "real" checksum error.

(This is all what bcachefs has done for years).


to post comments

There's a better solution than falling back to buffered io

Posted Nov 12, 2025 21:05 UTC (Wed) by Wol (subscriber, #4433) [Link]

Wasn't that benchmarked a few years back? Somebody disabled caching on file copies, and especially with larger files the uncached copy was maybe ten times faster?

Cheers,
Wol


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds