Putting the problem into perspective

Several years ago I ran into the problem of code trying forever for small allocations.
The fix of the immediate symptoms was: 96c7a2ff2150 ("fs/file.c:fdtable: avoid triggering OOMs from alloc_fdmem")

The basic issue was that the file table expansion code was making a 32KiB allocation. Which is the maximum size at which the code retries forever.

Except the code doesn't exactly try forever. Instead of failing the allocation the code instead triggers the OOM killer. Which in effect shifts where the failure was happening. In the case I was dealing with this caused the OOM killer to be triggerd on a system with roughly 4GiB free memory. The problem was that there were no chunks of memory of size 32KiB large and at that point it had no way to defragment the memory to make a 32KiB chunk of memory available.

The file table code in question had a fallback to handle a large page allocation failure. The code performs a vmalloc instead of a kmalloc.

Which demonstrates two things.
- That all allocations <= PAGE_ALLOC_COSTLY_ORDER won't fail because such pages will always be available is observably wrong.
- That there is actually harm in retrying forever on some code paths. As my fix demonstrated the retrying forever heuristic took a system that would have stayed up and caused it to crash.

That said I don't argue that on most code paths retrying forever is generally harmless as many times there isn't much that can be done except return an error to userspace.