Avoiding memory-allocation deadlocks

Posted Apr 16, 2014 15:57 UTC (Wed) by RelytDeveloper (guest, #96089)
In reply to: Avoiding memory-allocation deadlocks by ncm
Parent article: Avoiding memory-allocation deadlocks

I may not be fully qualified to answer your question, but Kernel.org uses the GIT change management control platform; and hosts several instances of cGIT, a web-based GIT wrapper.

The Kernel is big. It's probably too big to read, and likely too big to understand in one sitting.

Using techniques such as GIT Bi-Section, find, locate, grep, awk, and sed are necessary tools to manage such a large code base.

Since we did not always have GIT, records of the Kernel pre-dating GIT are spotty. A Git-Bi-Section compares the last known working code base to the regression code base, and tells you which Change-ID-hashes account for the trouble.

It is a professional quality GNU/Linux Kernel, but it spans many architectures and lieutenant-headed projects, and eight (?) versions.

Avoiding memory-allocation deadlocks

Posted Apr 19, 2014 7:51 UTC (Sat) by nix (subscriber, #2304) [Link]

ncm wasn't talking about the process of analyzing the problem -- he was talking about the somewhat chaotic and ad-hoc 'add this gfp flag, whoops, that's not enough, make it a pf, now add a mask to ignore it again' iterative process.

I'd say this is unavoidable, not so much because the solution space is so large as because the requirements change. Things like swap-over-NFS, suspend, early kmalloc etc imposed new requirements regarding things like allocator locks that simply weren't there before. e.g. before swap-over-networking turned up, you could safely chuck away network packets that arrived when under heavy memory pressure: they'd be retransmitted later and all would be fine. But if that might be part of the swapout process, suddenly you can't ignore that any more and the whole panoply of mm locks collides with the whole panoply of networking locks and the networking layer's own allocations and, well...

I'll admit that I'm amazed that anything got done at all before lockdep was around. Even with lockdep, locking hierarchy maintenance still feels terribly haphazard. There's really not much documentation even for crucial core locks like the mmap_sem (one comment partially documenting core parts of the locking hierarchy, that's all). You're just expected to know, and if you don't know and lockdep doesn't save you (e.g. if spinlocks are involved) expect a nice long fun crashy debugging session.