The possible demise of remap_file_pages()

By Jonathan Corbet
May 7, 2014

The remap_file_pages() system call is a bit of a strange beast; it allows a process to create a complicated, non-linear mapping between its address space and an underlying file. Such mappings can also be created with multiple mmap() calls, but the in-kernel cost is higher: each mmap() call creates a separate virtual memory area (VMA) in the kernel, while remap_file_pages() can get by with just one. If the mapping has a large number of discontinuities, the difference on the kernel side can be significant.

That said, there are few users of remap_file_pages() out there. So few that Kirill Shutemov has posted a patch set to remove it entirely, saying "Nonlinear mappings are pain to support and it seems there's no legitimate use-cases nowadays since 64-bit systems are widely available." The patch is not something he is proposing for merging yet; it's more of a proof of concept at this point.

It is easy to see the appeal of this change; it removes 600+ lines of tricky code from the kernel. But that removal will go nowhere if it constitutes an ABI break. Some kernel developers clearly believe that no users will notice if remap_file_pages() goes away, but going from that belief to potentially breaking applications is a big step. So there is talk of adding a warning to the kernel; Peter Zijlstra suggested going a step further and require setting a sysctl knob to make the system call active. But it would also help if current users of remap_file_pages() would make themselves known; speaking now could save some trouble in the future.

Index entries for this article
Kernel	remap_file_pages()

The possible demise of remap_file_pages()

Posted May 8, 2014 7:48 UTC (Thu) by kiryl (subscriber, #41516) [Link] (2 responses)

> But that removal will go nowhere if it constitutes an ABI break.

It doesn't break ABI. remap_file_pages() backed by nonlinear mappings is replaced by the patchset with an emulation which creates multiple VMAs. It's slower but not an ABI break.

The possible demise of remap_file_pages()

Posted May 9, 2014 13:52 UTC (Fri) by chrish (guest, #351) [Link] (1 responses)

PyPY's software transactional memory branch is a heavy user of remap_file_pages(). Not sure if this approach would still be practical with that slowdown...

The possible demise of remap_file_pages()

Posted May 9, 2014 14:30 UTC (Fri) by chrish (guest, #351) [Link]

Apparently, it would be good enough for PyPy. From Armin, on the PyPy-dev mailing list: "I've explained the PyPy situation, and it looks now like they're going to replace remap_file_pages() with an emulation based on mmap() and, I hope, take care of the issue that the mmap() solution is limited to 65536 by default (i.e., increase the default limit). This would be good enough for PyPy, where the calls to remap_file_pages() are themselves not highly performance-critical."

(https://mail.python.org/pipermail/pypy-dev/2014-May/01247...)

The possible demise of remap_file_pages()

Posted May 15, 2014 6:39 UTC (Thu) by weue (guest, #96562) [Link] (3 responses)

This is nuts.

Unless the "emulation" performs as well as the original API in both time and space (extremely unlikely), this patch should be sent directly to the trash bin where it belongs.

Breaking compatibility and removing features is never acceptable.

Have the GNOME 3 guys started an attempt to take over the kernel?

Anyway, the main and obvious use case on 64-bit are emulators using this to emulate virtual memory efficiently.

The possible demise of remap_file_pages()

Posted May 15, 2014 10:01 UTC (Thu) by yeti-dn (guest, #46560) [Link]

Have you ever maintained any complex software?

The possible demise of remap_file_pages()

Posted May 15, 2014 11:00 UTC (Thu) by tao (subscriber, #17563) [Link] (1 responses)

Hmmm.

Tough choice:

1.) Keep hard to maintain code (that has very few users and has alternative APIs to replace it) around forever, thus creating a maintenance burden

*or*

2.) Remove said code, provide a shim layer instead (that has lower maintenance cost but also slightly worse performance) and encourage the few users of the deprecated API to move to the alternate APIs

Oh wait, it's not a tough choice at all, nor is it very similar to the GNOME 3 philosophy.

Unless a feature being removed has:

a.) a lot of users (this is something that GNOME 3 ignores)
b.) isn't provided in other ways (this is also ignored in GNOME 3)
c.) no advance warning is given

I don't see anything wrong with removing a feature.

Here, there's:

a.) very few users of the API in question
b.) alternate ways to achieve it (and the use cases are very narrow anyway)
c.) plenty of advance warning plus a shim layer providing the old API for the time being

Thus I can only say: good riddance.

Also, note that my comparison to GNOME 3 above does not necessarily mean that I find GNOME 3 bad. I question a lot of the changes they've made (Nautilus has been turned into something I no longer use, a lot of plugins are needed to get the behaviour back in shape, the removal of the possibility to customise the toolbar in Epiphany made me switch to Firefox and too much is hardcoded these days), but when it comes to GTK+ and GLib they're definitely doing the right thing -- when they find that some of their old APIs are horribly broken, they deprecate them, then get rid of them. The newer versions of GLib and GTK+ are much cleaner than they used to be.

Imagine a world where shit like bcopy() and gets() (at least glibc now doesn't export it if_ISOC11_SOURCE is defined) would be removed from glibc. Ahhh, what a glorious thought.

The possible demise of remap_file_pages()

Posted May 21, 2014 13:25 UTC (Wed) by nix (subscriber, #2304) [Link]

There are actually some things you can do with remap_file_pages() that are just not implementable with other APIs without a fairly large performance hit: e.g. you can share a memory region with many other processes, then have any of those processes adjust its mapping *as seen by all the others*, atomically, because they all share the same VMAs: an emulation would have to adjust VMAs for every process in turn (unlikely to retain the atomicity semantics, for starters).

Now it may be that nobody depends on those semantics, but it was a *nice* IPC facility. It's just not a facility that anyone much uses :(