JVM default stack size and 64KB base pages [LWN.net]

JVM default stack size and 64KB base pages

Posted Jun 23, 2006 20:24 UTC (Fri) by vaurora (guest, #38407)
In reply to: KHB: Transparent support for large pages by ortalo
Parent article: KHB: Transparent support for large pages

I heard this story third-hand, so any corrections or additional details are welcome.

An unnamed large systems company decided to implement 64KB base pages in its OS. Everything looked dandy and great, except that for some reason nothing would work under Java. A closer look revealed that the default stack size of each thread in the JVM was set to 2*PAGESIZE... and the maximum stack size was 128KB. So threads would be created with 128KB stacks, immediately trigger a stack overflow, and die. This clearly is a bug and an abuse of PAGESIZE. However, given said company's fanatical devotion to strict binary compatibility above all else, it was decided that 64KB base page size could not be shipped because it would not work with that version of the JVM. This was an unpopular decision; it's possible it's been reversed since then.

The moral of the story is either "Strict binary compatibility is bad" or "Use transparent large pages and keep your base page size the same" or possibly "Computers are hard, let's go shopping!" (as Operating Systems Barbie is wont to say).

Other versions of this story probably exist for other OS's trying out larger base page sizes which undoubtably found the same bug; I don't know how they decided to handle the problem. But I'd love to hear the story!

-VAL

(Log in to post comments)

JVM default stack size and 64KB base pages

Posted Jun 23, 2006 21:37 UTC (Fri) by opalmirror (subscriber, #23465) [Link]

I designed and implemented a mechanism for a proprietary always-memory-resident RTOS to automagically split or join pages into naturally aligned clusters in the page table to make optimal use of the available TLBs and page sizes, implemented it on PPC4xx, and helped others apply it to other software page replacement CPU architectures from PPC, ARM/Xscale, MIPS, and SH. In that OS PAGESIZE stayed the minimum size but the OS looked at mapping requests and figured out how to minimize the use of TLBs. The use case for a memory-resident system and a demand paged one makes for very different design decisions though; for Linux one makes really different tradeoffs.

JVM default stack size and 64KB base pages

Posted Jun 24, 2006 11:52 UTC (Sat) by nix (subscriber, #2304) [Link]

I'm missing something. If the crash was due to the JVM unconditionally creating stacks of 2*PAGESIZE, then wouldn't the default (smaller) page size make the problem worse? (I doubt a JVM would work well with 8Kb or 16Kb thread stacks!)

Or were they doing something like

if (PAGESIZE > 16384)
... size stack to 2*PAGESIZE ...
else
... size it to something sane ...

If so, well, *ick*.

JVM default stack size and 64KB base pages

Posted Aug 4, 2006 0:23 UTC (Fri) by tpepper (guest, #31353) [Link]

I don't know the full details (heard it 3rd hand like Val) exactly but as she says there was an assumption inside the JRE apparently about what page size was, they were allocating a certain number of pages thinking their stack would fit exactly in those pages and their math to traverse memory then was wrong when the pages were larger (ie: 64k instead of 4k). I thought it was a 12k stack using three 4k pages. Presumably they then got three 64k pages, with their 12k assumed stack falling entirely within the first if accessed straight from the start. Accesses to parts of the stack via the pointers to the second and third pages wouldn't actually then find their stack's data/frames...It's easy to imagine people doing unusual optimisations based on assumptions about page size.

At any rate they figured out how to work around it with an LD_PRELOAD option for their existing code pending an update.