Is it just me who's wondering whether all this complexity is worth it?
What was it needed for again? To have a reverse mapping? Isn't that mostly useful for swapping and not much else? It seems silly to always add the overhead when it's almost never needed. Why not only have the overhead when actually swapping or whenever it's needed? Then this whole mess disappears from the common case and the complex stuff can be separated and isolated.
I have to read more code and think more about it, preferably with a clear mind.