So, I guess I don't understand how the kernel is involved with managing the stack for user space processes? It obviously is, but what is the mechanism for it?
Hmmm... the process starts with some stack at the top of memory, which it can push/pop data from (calling to and returning from function calls). If when the process pushes more data onto the stack than is actually allocated for it (i.e. it goes over the page boundary), this will fault to the kernel, which automatically allocates more memory for it? Does that sound about right?
The bug is that if the stack runs into some already allocated heap pages, then no fault occurs (since the memory is already allocated), and so now you can write to the heap area and effectively control the stack and return addresses on it.
So, Linus' fix is to add a guard page at the bottom of the currently allocated stack? And this is causing some problems with some applications that probably pay way to much attention than they should to the stack allocation.
Seems like this would be a REALLY old problem, that up until recently was effectively impossible to exploit (due to the sizes of memory and in ability to get a remote process to actually allocate that much memory).
Separately, how are other Unixen not vulnerable to the exact same problem? If not, how do they address the problem?