The problem of unnecessary readahead [LWN.net]

By Jonathan Corbet
April 18, 2025

The final session in the memory-management track of the 2025 Linux Storage, Filesystem, Memory-Management, and BPF Summit was a brief, last-minute addition run by Kalesh Singh. The kernel's readahead mechanism is generally good for performance; it ensures that data is present by the time an application gets around to asking for it. Sometimes, though, readahead can go a little too far.

Singh had a couple of example cases; the first had to do with ELF binaries that include extra padding for compatibility with systems that have a large base-page size. On systems with smaller pages, that padding is just useless data that could happily remain on disk, but the readahead machinery is reading it into the page cache anyway. That creates useless I/O and wastes memory with data that nobody will need. Another example was APK (Android package) files, which contain a combination of compressed and uncompressed data in the same file. Readahead is bringing in the compressed data, even though it will never be read.

In both cases, Singh said, readahead is bringing in unwanted data, even if it is outside of the part of the file that has been mapped. fadvise() can be used to prevent the readahead, but that affects the entire file, including the parts where readahead is beneficial. Singh has also tried placing guard pages in the memory area where the file is mapped, but they do not stop readahead. David Hildenbrand suggested that having the readahead mechanism actively look for guard regions would be a good idea, but this checking might prove to be expensive.

A participant asked why the whole file, including the padding, is being mapped; Singh said that the alternative would create a lot more virtual memory areas (VMAs), which would have a performance cost of its own. Hildenbrand suggested that madvise() could be used to force the reading of the desired part of the file. The memory of that operation would be lost, though, if the populated pages are reclaimed. This approach could also slow down application load time, he said.

Singh had one other possibility to discuss: optimizing the handling of sparse files. The holes in files could be marked in the kernels and mapped to the zero page if accessed by the process. There would be some pitfalls; there are complications if the zero-page is modified (causing a copy-on-write operation) while pinned into memory for example. It would also complicate the handling of files that have been mapped shared; perhaps, he said, that case does not need to be supported.

As the session (and the conference) ended, Hildenbrand suggested that efforts to optimize for unexpected cases would be misplaced. Perhaps, he said, it would be sufficient to just stop readahead whenever a hole is reached in the file.

Singh has posted the slides from this session.

Index entries for this article
Kernel	Memory management/Readahead
Conference	Storage, Filesystem, Memory-Management and BPF Summit/2025

to post comments

Last minute, indeed

Posted Apr 18, 2025 21:27 UTC (Fri) by willy (subscriber, #9762) [Link] (14 responses)

The biggest problem with this session is that nobody told me. Instead, I was in the filesystem room where we were discussing a solution to this problem. Sorry the MM people wasted their time.

Last minute, indeed

Posted Apr 19, 2025 11:11 UTC (Sat) by jlayton (subscriber, #31672) [Link]

I would have liked to have heard this too. This seems like it should have been a combined FS+MM session.

Last minute, indeed

Posted Apr 19, 2025 15:59 UTC (Sat) by hossbeast (subscriber, #137045) [Link] (1 responses)

What is the solution? (high level)

Last minute, indeed

Posted Apr 22, 2025 15:56 UTC (Tue) by willy (subscriber, #9762) [Link]

Lift the hole/data information from the filesystems to the VFS. I've described this before on fsdevel, but certain senior filesystem developers think it's impossible. Ted didn't seem inclined to agree that it was impossible, so the only way to find out is to try.

This doesn't help network filesystem clients, but that's not where the problem lies right now. It does help NFSd as it also needs this information to implement READPLUS.

Last minute, indeed

Posted Apr 19, 2025 16:01 UTC (Sat) by surajm (subscriber, #135863) [Link] (10 responses)

It's almost like synchronous in person conversation as the primary means to discuss problems is perhaps inadequate for such a large and diverse developer base. I'm sure there exists many more individuals who could usefully contribute if there existed more asynchronous globally friendly mechanisms. Email is also not the necessarily the solution here.

Last minute, indeed

Posted Apr 20, 2025 8:43 UTC (Sun) by mcgrof (subscriber, #25917) [Link] (2 responses)

Has anyone even evaluated if it makes sense for readAhead for fast SSDs?

Last minute, indeed

Posted Apr 20, 2025 20:59 UTC (Sun) by khim (subscriber, #9252) [Link] (1 responses)

Give the fact that focus seems to be on Android fast SSDs are given.

Although it's probably more important for cheap phones, but even cheap phones have pretty fast SSDs, these days.

Last minute, indeed

Posted Apr 21, 2025 6:28 UTC (Mon) by surajm (subscriber, #135863) [Link]

sdmmc and ufs based storage found on most phones is different from SSD. That said, it's still nand flash based media with similar latency characteristics. Read ahead still makes a lot of sense in the context of flash based media, otherwise you'd have 50us+ delays on every page fault.

Last minute, indeed

Posted Apr 21, 2025 11:48 UTC (Mon) by Wol (subscriber, #4433) [Link] (6 responses)

> It's almost like synchronous in person conversation as the primary means to discuss problems is perhaps inadequate for such a large and diverse developer base.

"Synchronous in person is the worst form of communication - apart from all the others ..."

How are you going to improve on in-person communication? The problem here - and everywhere else - is if you don't know about it you can't join in, and if you find out after the event it's too late.

EVERY form of alternative suffers from the exact same problem - if you don't know about it in time you're stuffed (which is catch-22 - that itself is a communication problem), and many of the alternatives also suffer from the internet problem - as soon as you unlimit attendees the signal/noise ratio plummets - be it trolls or just well-meaning people who don't know what they're talking about.

Cheers,
Wol

Last minute, indeed

Posted Apr 23, 2025 7:26 UTC (Wed) by taladar (subscriber, #68407) [Link] (5 responses)

Except that other forms of communication do not suffer from the same problems.

Synchronous communication of any kind always suffers from the issue that you might not be able to attend even if you know about it because you have conflicting plans for that time.

In-person communication is even worse here since it only allows you to attend when you have enough time to travel to the location and back, not to mention limitations on travel due to e.g. health concerns, passport related concerns, financial concerns,...

Verbal communication (as most in person communication is) also suffers from the problem that the discussion is often not recorded so important points are lost. People also might not listen while they think about what to say themselves next or might miss something that is obvious in written communication where you can just re-read what someone else said.

Synchronous communication also suffers from the issue that inevitably only one thread of communication can be followed even if someone brings up multiple topics that are worth discussing.

The problem you mention of limiting the participants on the other hand is completely orthogonal to the discussion format.

Benefits of different forms of communication

Posted Apr 23, 2025 9:53 UTC (Wed) by farnz (subscriber, #17727) [Link] (4 responses)

Part of this is that we're under-trained in communicating well; in-person communication is the form of communication with highest demands on the people communicating, but also allows for the fastest information transfer. Next fastest is video-enabled conference calls, then voice-only calls, then synchronous text chat, with async text chat and e-mail as the slowest way to transfer information between two co-operating parties.

However, we tend not to be trained on getting the most out of different communications options. There's a lot that needs to be learnt to make good use of synchronous communications:

Setting agendas, and enforcing them; how can you know who should be in a meeting if you don't know what the topics are? And how can you know that a meeting can be safely skipped if you don't know what'll be discussed there?
Stopping meetings if the right people aren't present; there's no point having a detailed discussion if the experts on the topic aren't present.
Getting the right people to attend a meeting; time zones, travel complications (if in-person) and other things can result in the right people not wanting to attend, and you need to get them to join to make it productive.
How to prevent loud/present from dominating over expertise - we've all been in meetings where the noisiest person takes over, even though the expert is quietly sitting in the corner.
Splitting off side-discussions that don't need to be synchronous (or that don't need all participants present). Why have 10 people listen to two people discuss details (when they can chat in the hallway later), or spend expensive time on a synchronous chat that would be better as a series of back-and-forth e-mails?
Providing a summary of the meeting (like this article) after the fact, so that people not present can know what was said and why.
Halting low-productivity low-effort communication and setting up a higher-effort meeting. Why have a big back-and-forth over e-mail when a video call with screen sharing resolves the issue quickly?
Appropriate ways to pre-communicate positions on agenda items, without having a pre-meeting or having an intense discussion in a slow communication medium, while still allowing the item to come off the agenda if someone's position resolves all the points immediately. Reserve the meeting itself for things that need the high-throughput communication, using low-effort communications for stuff everyone agrees on.

And that's just a summary of the basics you need to know to make effective use of synchronous communications. None of those items are easy, and at least 3 of those bullets are things that upset the people who prefer synchronous communication to async comms - often bad managers and the like.

Benefits of different forms of communication

Posted Apr 23, 2025 10:37 UTC (Wed) by Wol (subscriber, #4433) [Link] (2 responses)

> that upset the people who prefer synchronous communication to async comms - often bad managers and the like.

Or people who just prefer face-to-face full stop ... (like me).

Cheers,
Wol

Benefits of different forms of communication

Posted Apr 24, 2025 11:02 UTC (Thu) by taladar (subscriber, #68407) [Link]

Actually that is also part of the problem. Everyone tries to get the communication to happen in the format they prefer.

This is probably not malicious in open source projects that often but in e.g. office or general politics there might certainly be an agenda behind it, e.g. someone with bad arguments but who is very charismatic is able to more easily suppress unwanted counter-arguments in an in-person discussion than in a written one. Or someone would prefer not to have a written record of the things they said in a particular discussion.

Benefits of different forms of communication

Posted Apr 24, 2025 11:48 UTC (Thu) by farnz (subscriber, #17727) [Link]

That's a huge part of the problem; face-to-face is high effort, high cost. Part of being good at communication is knowing when paying that price is a huge benefit, and when it's wasteful.

For example, it's worth paying the price of face-to-face to reduce 2 weeks of async back and forth between 15 people resulting in hundreds of messages (whether via IRC or e-mail) to a 10 minute discussion resulting in a 25 line message that communicates the original positions of all parties plus the outcome of the meeting. It's not worth consuming 10 minutes of your time to have a face-to-face meeting when the entire meeting can be reduced to two e-mails of 1 line each.

Effective communication is not simple; you need to match the cost of communicating a certain way to the benefit you get from a given chunk of communication. And that, in itself, is a tricky problem.

Benefits of different forms of communication

Posted May 2, 2025 1:20 UTC (Fri) by fest3er (guest, #60379) [Link]

I'll interject two points pertinent to human interaction, things I've been saying for years, decades even. That we will never perfect inter-human communication is not a reason not to try.

First, the three most important things everyone in every society must do every day are to

communicate,
communicate, and
communicate.

Good communication is the lubricant that keeps the cogs of society turning smoothly; poor communication throws shoes into the gears. Each of us must communicate our thoughts as clearly and succinctly as possible. We all have to—or had to—learn to slow down our thinking enough to put our thoughts and ideas into coherent words, sentences and paragraphs and, later, to record them as black marks on white media.

Students of history might find a correlation between the rapid development and implementation of new methods of communication with the rapid development of society. The easier it is for people to share thoughts, the faster society grows and advances.

Second, human languages are programming languages. Right now, I am attempting to program your neural networks to think my thoughts. Your responses to this post will be your attempts to program my neural network to think your thoughts.

We all know we must carefully write computer programs lest they destroy data or produce incorrect results. The same goes for human programs. We must carefully compose our writings to others who need to know so that they accurately communicate our thoughts (don't destroy or mangle our thought data), minimize misunderstandings (don't produce incorrect results), minimize back and forth "Do you mean X? Y? Z?" queries, and accomodate "What about A? B? C?" queries. As many of you have noticed, this is the nature of developing and implementing standards: slow, tedious processes that are easily derailed by people who have their own (potentially orthogonal and/or incompatible) agendas.

Organizing and communicating thoughts and ideas is a slow, tedious process for which there are no shortcuts. An added benefit to such labors is that proficiency in preparing human programs directly translates to proficiency in preparing computer programs (which are inherently less complex). That which is well communicated is the easiest to understand.

As with all imperfect human endeavors, there is always room to improve inter-human communication. Even here on LWN.

And now, back to our regularly scheduled program....