Kernel feedback for GC (was: The Managed Runtime Initiative)

One thing I once thought about many years ago was a way to provide feedback to a userspace GC from the kernel as to how much memory it should use. If the kernel was under memory pressure it would tell the userspace GC to discard some pages. On the other direction, the userspace GC would tell the kernel about how many pages it expected to be able to free easily, and how many pages it expected to be able to free with a bit more effort (running a slower memory compaction, for instance).

The idea was to allow userspace programs to use the whole memory as a cache (for instance, for Firefox's memory cache) without the risk of going into swap. Also, if there were several GC-using userspace programs (for instance, several JVMs), they would be able to share the memory instead of fighting for it.

The API would be something that you would mmap() in your address space, some way to tell that something to free a page in that mapping (turning it into the shared copy-on-write zero page), some way for the program to tell the kernel how many pages it wanted to use for that memory area (a minimum value below which it cannot free anymore, an optimum value, and a maximum value above which it has no use for the memory), and some way for the kernel to notify the program about the desired usage levels for that memory area (minimum, optimum, and maximum values). As you can see, I did not work out the API details.