Expanding the kernel stack

Posted May 30, 2014 18:41 UTC (Fri) by PaXTeam (guest, #24616)
In reply to: Expanding the kernel stack by luto
Parent article: Expanding the kernel stack

i have good news then, the very new GRKERNSEC_KSTACKOVERFLOW feature solves this without breaking up huge pages. throw in our 3-year-old move of thread_info off the kstack and we've got a winner! ;)

Expanding the kernel stack

Posted May 30, 2014 19:55 UTC (Fri) by luto (guest, #39314) [Link] (3 responses)

That thing uses vmalloc, right? Then it still prevents hugepages from being used for the kernel stack. This will cost a TLB slot or two. It's probably still a good tradeoff.

Expanding the kernel stack

Posted May 30, 2014 20:49 UTC (Fri) by PaXTeam (guest, #24616) [Link] (2 responses)

correct but note that the lowmem map will keep using 2M/1G pages. as for its TLB impact, it's a tradeoff between one 2MB (or 1GB) TLB entry and 1-4 4k entries. the net performance impact depends on how the TLBs for each page size are organized in a given CPU and the access pattern of the virtual memory mapped by those entries (e.g., if there're separate TLBs for each page size and the access pattern continuously exhausts one but no the other(s) then obviously freeing/taking up extra entries will have a net positive/negative impact). i think in practice it'll come down to how many accesses are made to lowmem vs. the vmalloc ranges in a workload.

another advantage is that vmalloc by its nature handles lowmem fragmentation much better which becomes even more important now that amd64 kstacks have become order-2 allocations. it'd also be easy to implement lazy page allocation for kstacks further reducing their memory consumption (let's face it, many kstacks will never actually make use of the whole 16k yet they'll always have to be fully allocated in the current scheme).

Expanding the kernel stack

Posted May 30, 2014 20:52 UTC (Fri) by luto (guest, #39314) [Link]

It could pay to send the patch upstream. If it's clean, I'll advocate for it.

Expanding the kernel stack

Posted Jun 27, 2016 0:50 UTC (Mon) by luto (guest, #39314) [Link]

Lazy page allocation is an interesting idea. I can think of two potential problems, though:

1. What do you do if lazy allocation fails?

2. Hitting a not-present page on the stack is likely to result in a double-fault. Intel's manual advises against trying to recover from a double-fault, and I'd like to know why before messing with it. Even if recovery were guaranteed to work, it could be interesting trying to allocate memory (which can block) in a double-fault handler.

The espfix64 code can double-fault and recover, but we ran that specific abuse of the CPU by some Intel and AMD engineers before doing it.

Expanding the kernel stack

Posted Jun 1, 2014 8:35 UTC (Sun) by richard_weinberger (subscriber, #38938) [Link] (2 responses)

Are you willing to send this feature as stand-alone patch upstream for review?
Would be awesome. :-)

Expanding the kernel stack

Posted Jun 1, 2014 12:38 UTC (Sun) by PaXTeam (guest, #24616) [Link] (1 responses)

it's spender's code, so you'd have to convince him to go through the process.

Expanding the kernel stack

Posted Jun 2, 2014 11:49 UTC (Mon) by dgm (subscriber, #49227) [Link]

Or someone else to adopt the code and champion it.