Quotes of the week [LWN.net]

Patch authors are expected to proactively look into the AI-generated reviews and handle such feedback as any other kind of review: either debate it or address it. In both cases a reply on the mailing list is expected.
Authors are strongly encouraged to run LLM reviews on the posted patches in advance of the actual post. Large series triggering a significant amount of AI-generated feedback will likely get little attention from maintainers and reviewers.

— Paolo Abeni updates policy for the networking subsystem

Back when Linux ran on an abacus, I added __GFP_NOFAIL because four or five sites (file systems) were infinitely looping on alloc_pages until it succeeded. I figured we should move this operation into the page allocator so we could easily find those sites and fix them to Not Do That.
I made it very clear (changelog and code comments) that no new code should use __GFP_NOFAIL. How did that work out?
    hp2:/usr/src/linux-7.2-rc4> grep -r __GFP_NOFAIL | wc -l
    241
So much lameness!

— Andrew Morton

I'll also be up front about the fact that this patchset is based entirely on LLM predicted code. I review it all and correct any errors, issues, concerns, etc immediately as each new hunk of code is predicted by the LLM to prevent errors from compounding and multiplying. Thus the code ends up the way I'd write it manually, not the way an unchecked LLM would vibe code it. Mistakes in the code are mine, not a result of an LLM being used to predict the code I'd write faster than I can write it myself.

— Dave Chinner

to post comments

So much lameness!

Posted Jul 30, 2026 9:00 UTC (Thu) by neilbrown (subscriber, #359) [Link] (13 responses)

The lameness here is that __GFP_NOFAIL isn't the default. Imagine how much unexercised error handling code that would obviate!

When does GFP_KERNEL ever fail rather than wait and retry?
I think it might if the OOM killer has killed the process, but I don't think that is fundamentally needed.

(Yes, I know, I should submit a patch....)

So much lameness!

Posted Jul 30, 2026 14:25 UTC (Thu) by daroc (editor, #160859) [Link] (6 responses)

There are a few reasons. One of the most important ones is that sometimes allocating memory can require flushing file caches or other pending operations to disk ... and on some filesystems that can require allocating memory, leading to deadlock. There are other problems, though, including allocations that are only used for caching/performance, and can safely be omitted. Also, the kernel can try to allocate more memory than the device actually has, and that will always fail just for hardware reasons, so you need to handle allocation failures anyway.

I think a kernel that was built from scratch around the assumption that allocations simply don't fail could be made to work. But getting the Linux kernel into shape to make that practical would be an enormous effort.

So much lameness!

Posted Jul 30, 2026 21:12 UTC (Thu) by neilbrown (subscriber, #359) [Link] (5 responses)

Thanks for your thoughts, but I don't think any of that has direct bearing on the issue at hand.

Certainly deadlocks must be avoided and for that we have GFP_NOFS and GFP_NOIO. Using these (or the memalloc_FOO_save() equivalent) is important but doesn't materially affect WHETHER memory can be found, only WHERE it is likely to be found .

Certainly some allocations can safely fail, and for that we have GFP_RETRY_MAYFAIL or even GFP_NORETRY, and using those correctly where appropriate is certainly important.

I'm focused on GFP_KERNEL allocations which, in practice, (almost) never fail (if it did you would get a warning in the kernel log - have you ever seen one?). They are used when the caller really wants some memory to perform some task. In many cases the memory will be freed shortly afterwards. In other cases it will be added to a cache from where it can be freed whenever memory pressure requires it. These allocations generally won't have any chance of exhausting memory.

When performing GFP_KERNEL allocations (even with NOFS or NOIO in effect) the kernel can:

find some memory
wait for memory to become available
write something out to swap/filesystem
kill some process that is hogging too much memory
wait some more
try again

there is no benefit to anyone to fail in this case. If the kernel gets to a state where GFP_KERNEL allocation cannot succeed, then something is very deeply wrong and failing allocations rather then blocking is unlikely to make a noticeable difference.

So much lameness!

Posted Jul 31, 2026 9:51 UTC (Fri) by taladar (subscriber, #68407) [Link] (1 responses)

Maybe the "can't fail" goal is the wrong way to think about the goal. Many systems have two types of failures (e.g. HTTP with the 4xx and 5xx families or SMTP) where it only makes sense to retry on specific kinds of failures. Maybe the kernel could eliminate just that kind and still return failures in situations where a retry is genuinely not a sensible option?

So much lameness!

Posted Aug 1, 2026 6:42 UTC (Sat) by neilbrown (subscriber, #359) [Link]

Many systems have two types of failures

Internally the MM has that sort of distinction, such tracking whether there is any point triggering more writeback or clearing more caches, or whether it is time to go for the bigger hammer and trigger the OOM killer. And to some extent we can ask page_alloc() to fail at different points - __GFP_NO_RETRY or __GFP_RETRY_MAY_FAIL or __GFP_NOFAIL. And these are important (though I'm not entirely sure how how to choose between the first two of those).

But the thing that will make the big difference to code is to make "no fail" a supported default for a large classes of use cases. "no fail" is enormously different from "might fail but very unlikely" when considering the boiler-plate error handling code you have to wrap around each call.

So much lameness!

Posted Jul 31, 2026 13:14 UTC (Fri) by daroc (editor, #160859) [Link] (1 responses)

Hmm. I guess it makes sense that, in most contexts, waiting for more memory to become available is a more sensible option than failing. And I certainly agree that eliminating seldom-encountered error paths will make things simpler and more reliable, and should be pursued.

If having infallible allocations is clearly simpler, and we've added more infallible allocations over time, that does make me wonder why the MM maintainers are opposed to the idea. Unfortunately, beyond the basic understanding I expressed in my previous comment, I'm not sure why that would be. I suppose I'll have to go dig into the discussions in the mailing list when I have a chance.

So much lameness!

Posted Aug 1, 2026 6:34 UTC (Sat) by neilbrown (subscriber, #359) [Link]

that does make me wonder why the MM maintainers are opposed to the idea.

It is perfectly normal for producers to have a vastly different perspective and drive to consumer, which is why customer feedback is sought so much (and when you have finished reading this comment, please stay on the line and answer one quick question), and why events like plumbers and lsfmm are so important.

But unfortunately most customers can't be bothered giving feedback (I bet not one of you stay on the line!) when it is much easier to just use that flag we were told not to use, or write a simple retry loop that is "good enough". We even have a name for this. It is the "platform problem" which refers to people working around weaknesses in the platform (Linux core MM code in this case) rather than working to get them fixed.

So it isn't the MM maintainers that I find fault with, it is the rest of us (particularly me) for not raising the need clearly enough or often enough to get a proper conversation going. And I'm quite heartened by Willy's report that __GFP_NOFAIL was discussed at lsfmm. We can only expect the MM maintainers to make a change if we ask them to (and only accept a patch if we send it).

So much lameness!

Posted Jul 31, 2026 18:30 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

Interestingly, the Windows kernel also in the "can't fail" camp. Drivers are designed to pre-allocate pools during the initialization time so that allocations can't fail when the IRQL is raised above the default level. RAM overcommit is also not a real thing in Windows, every app is supposed to have enough backing RAM/swap storage.

Linux was designed with heavy overcommit in mind to work better in constrained environments. There was an extensive discussion on Usenet about this, but I can't find it anymore because Google Groups suck.

But now RAM is (relatively) abundant, so it looks like Windows' solution may be better. Just reserving some RAM for emergency allocations might be a much better option than having never-exercised error recovery paths.

So much lameness!

Posted Jul 30, 2026 22:36 UTC (Thu) by willy (subscriber, #9762) [Link] (5 responses)

Some fun corner cases ...

kcalloc(1UL<<33, 1UL<<33, GFP_KERNEL);

That must fail. The requested size is larger than SIZE_MAX.

kmalloc(1UL << 50, GFP_KERNEL);

That must fail. That's larger than the amount of memory supported by any CPU on the market today.

vmalloc(1UL << 45);

That must fail on x86-64. It doesn't fit in the amount of space allocated to vmalloc space.

for (i = 0; i < 1 << 30; i++)
for (j = 0; j < 1<< 20; I++)
folio_alloc(GFP_KERNEL, 0);

At some point, one of these allocations must fail. The aggregate is larger than the amount of memory in the machine. But no individual allocation is too large to fail.

There is an alternative to failing, and that's putting the task to sleep forever. I don't think that's a better alternative than failing.

So much lameness!

Posted Jul 31, 2026 0:12 UTC (Fri) by neilbrown (subscriber, #359) [Link] (4 responses)

Yes, large allocations have to fail. So instead of redefining GFP_KERNEL, add GFP_SMALL which BUG()s if the size is larger than some agreed value (8k? 16k? __GFP_NOFAIL is already limited to 2 pages I think) but never ever fails. Then encourage its use.

vmalloc() has assorted reasons for failing. But I don't think anyone seriously needs __GFP_NOFAIL for vmalloc()

Allocating a billion pages sequentially is a bug just like division by zero is a bug. Don't write bugs. If people let bugs get into the kernel, then bad things can happen.

Putting the process to sleep forever (possibly by calling BUG()) means the caller never has to handle errors that never happen in practice. This is a win for code clarity. Allowing a known-buggy process to continue running with a NULL return is just asking for more bugs to trigger.

My main point is not that GFP_KERNEL allocations of modest sizes shouldn't fail. It is that they DON'T fail. Given that, let's be open about it and remove all the error handling noise and stop shaming people who are using the right tool for the job.

Alternately, someone can tell me that I'm wrong and spell out the circumstances where small GFP_KERNEL allocations do fail. Then we can document it and maybe discuss if it is reasonable to fail in those circumstances. But no-one has done that yet (except the OOM-killed processes which was years ago and may or may not still be true).

So much lameness!

Posted Jul 31, 2026 0:26 UTC (Fri) by willy (subscriber, #9762) [Link] (3 responses)

I would suggest that focusing on ensuring that slab allocations never fail might be the right path to happiness. That way we have a fixed size and don't need to worry about stupid allocation sizes.

As far as the folio_alloc() example goes ... I did this to myself without writing a line of code. I forget exactly what I was doing (it was probably 20 years ago). I set up a ramdisk of some description and wrote to it. It gradually allocated more and more memory. Killed a bunch of processes, but as more memory became available, the ramdisk allocated it.

Eventually I think I had to reboot. But everything got slower and slower up to that point.

The size limit on GFP_NOFAIL was discussed at lsfmm. It's actually a problem because of drives with block size larger than PAGE_SIZE.

So much lameness!

Posted Jul 31, 2026 2:16 UTC (Fri) by neilbrown (subscriber, #359) [Link] (2 responses)

yes to slabs only. But kmalloc doesn't make it obvious which sizes are served with slabs (2 pages or less!).
no to configuring ramdisks or tmpfs filesystems which aren't limited to a small fraction of total memory (plus swap if they can use it)
When there is a specific use case that is problematic (large disk blocks) I'm in favour of mempools. Maybe they should use __GFP_RETRY_MAYFAIL rather than __GFP_NORETRY, but there should be minimal overhead except when memory is tight, and then at least they make progress. But that can be hard to work with.... However the lsfmm discussion would appear to support my position.

So much lameness!

Posted Jul 31, 2026 2:35 UTC (Fri) by willy (subscriber, #9762) [Link] (1 responses)

I didn't mean kmalloc allocations that happen to be satisfied by the slab allocator. I meant callers of kmem_cache_alloc(). Those should just never fail.

I hate mempools. We waste so much memory in them and they're such an ugly interface to use. I feel there must be a better way.

So much lameness!

Posted Jul 31, 2026 7:06 UTC (Fri) by neilbrown (subscriber, #359) [Link]

We waste so much memory in them

True. Anyone who requests more than 2 entries in the pool probably doesn't understand how they work. I might have been guilty of that once.... Does ras_log_ring_sw_init() REALLY request over 40,000 !?!?!?! Have I misread that?

I feel there must be a better way.

We have GFP_HIGH and GFP_MEMALLOC. When doing swap-over-NFS we used GFP_MEMALLOC rather than mempools on the network side because that was requested. That is a less ugly interface, but is also much less precise so it likely wastes (or spends?) even more memory. We really don't know how much memory we need for GFP_MEMALLOC allocations. At least with mempool we only reserve memory for subsystems that are active. For GFP_MEMALLOC we allocate some arbitrary amount based on how much RAM we have, not based on what we are doing with it.

I don't know that GFP_MEMALLOC is much good for multi-page allocations though. mempools would be fine for multi-page allocations.