The Emacs dumper dispute [LWN.net]

The Emacs dumper dispute

Posted Nov 30, 2016 19:08 UTC (Wed) by madscientist (subscriber, #16861)
Parent article: The Emacs dumper dispute

I'm confused by this article.

So, in Jan 2016 Emacs devs switched to a different implementation for dumper. Presumably this different implementation (is that one of the ones described in this article? Or something different?) works well with both old and new glibc implementations... doesn't it?

Then it was (as I understand it) discovered that if you had an older Emacs without the different implementation but installed a newer glibc without the old hooks, then a older, slower, unstable interface "ralloc" (whatever that is) would be used.

So the solution is to... create more new interfaces, which will be released in even newer versions of Emacs, which still won't help anyone running an older Emacs without these new interfaces? Why is that an improvement?

And, since Emacs 25.1 was released just over two months ago, why is that version also negatively impacted, if it had the new implementation decided on in January?

Sorry for being dense but I think I'm missing something fundamental here.

to post comments

The Emacs dumper dispute

Posted Nov 30, 2016 23:58 UTC (Wed) by nix (subscriber, #2304) [Link] (2 responses)

When the malloc_get/set_state() functions are not available, Emacs falls back to a built-in malloc(), which is slower and less efficient than glibc malloc(), but works (or was believed to until this ralloc() issue turns up). malloc_set_state() -- used when loading a previously-dumped Emacs -- *still exists* in newer glibc, but as a compat symbol that cannot be linked against: glibc will detect when the old heap format is used by something calling malloc_set_state() (i.e., by an Emacs linked against an older glibc) and will mark the old heap dynamically as something allocated but unfreeable, which is good enough to work with a newer glibc no matter what its malloc implementation (but unlike in older glibc, the dumped heap remains a separate arena which the new malloc() is ignorant of, so free()ing from it does nothing, and new malloc()s will never come out of that arena). However, newly-compiled Emacsen will obviously not get a heap format compatible with this scheme, so malloc_get_state() is not available even as a compat symbol (or, rather, it is, because Emacs still has an undefined symbol referring to it, but its implementation has been stubbed out and always returns -ENOSYS).

I'll have to catch up with the emacs dev list, clearly -- I had no idea about this latest ralloc mess, though it was probably predictable because next to nothing has used ralloc for years so it has probably bitrotted catastrophically.

The Emacs dumper dispute

Posted Dec 3, 2016 0:35 UTC (Sat) by giraffedata (guest, #1954) [Link] (1 responses)

I don't know what any of that has to do with the question, which was a very good one.

But I combed the article again, and see the answer:

The problem being solved by the portable dumper work is not that old Emacs won't run on new Glibc. The article starts off describing that problem, but only as an example of the effect of the larger problem:

So something clearly needs to be done to make the Emacs dumping facility more stable and, preferably, more maintainable going forward.

The article doesn't mention any solution for the Emacs 25.1-on-glibc-2.24 problem; maybe there isn't one.

The Emacs dumper dispute

Posted Dec 8, 2016 17:53 UTC (Thu) by nix (subscriber, #2304) [Link]

Er, the entire comment directly relates to what happens when a newly-compiled Emacs starts on a new glibc (it uses its own malloc()) and/or what an Emac compiled against an older glibc does (Emacs binds to the malloc_set_state() compat symbol, the implementation of which detects the old glibc's heap format etc etc).