The Emacs dumper dispute [LWN.net]

The Emacs dumper dispute

Posted Dec 1, 2016 6:50 UTC (Thu) by rsidd (subscriber, #2582)
Parent article: The Emacs dumper dispute

I'm extremely ignorant of this stuff, but if emacs depends on these low-level glibc hooks for loading the dumped image, how does it work on other platforms (OS X, Windows, the BSDs, various proprietary Unixes, etc?)

to post comments

The Emacs dumper dispute

Posted Dec 2, 2016 12:23 UTC (Fri) by nix (subscriber, #2304) [Link] (4 responses)

On those platforms, Emacs uses its own malloc() implementation, which is an ancient malloc from the early 90s and is notably less efficient than glibc's malloc() (which is itself not as good as modern malloc()s, but which has been unable to improve because almost any change would break previously-dumped Emacsen: hence all this mess).

(Needless to say, Emacs should not be providing its own not-terribly-good malloc() implementation. It should rely on the documented malloc() interface, sans ridiculous dumping hooks, and serialize for dumping. That Eli doesn't understand that this is obviously the only even vaguely maintainable approach is baffling to me.)

The Emacs dumper dispute

Posted Dec 2, 2016 14:09 UTC (Fri) by rsidd (subscriber, #2582) [Link]

Thanks for the explanation. Looks like things would have been better all around if Emacs had adopted the same strategy on linux and let glibc do its thing separately! But both of them being GNU projects complicates things :(

The Emacs dumper dispute

Posted Dec 4, 2016 9:25 UTC (Sun) by eliz (guest, #94829) [Link] (1 responses)

> On those platforms, Emacs uses its own malloc() implementation, which is an ancient malloc from the early 90s

This is factually incorrect: Emacs only uses its own malloc implementation during dumping. The dumped emacs executable uses the native malloc implementation on all the platforms mentioned in the previous post.

> It should rely on the documented malloc() interface, sans ridiculous dumping hooks, and serialize for dumping. That Eli doesn't understand that this is obviously the only even vaguely maintainable approach is baffling to me.

In fact, Eli understands this simple fact very well. There's never been any question that unexec should go. The issue at hand is what is the best alternative to unexec, with all its problems.

The Emacs dumper dispute

Posted Dec 8, 2016 17:55 UTC (Thu) by nix (subscriber, #2304) [Link]

> This is factually incorrect: Emacs only uses its own malloc implementation during dumping. The dumped emacs executable uses the native malloc implementation on all the platforms mentioned in the previous post.

Really? Last time I tried to trace through the bugger it spotted the lack of malloc_set_state() and fell back to gmalloc for everything from dumping onwards, with fairly nasty consequences for memory usage. If the nasty consequences only relate to the relatively small amount of state stored during dumping, I'm fairly mystified as to how multi-hundred-MiB variances could be so apparent -- but Emacs is such a dynamic system that maybe I was being misled by perfectly normal run-to-run variability.

The Emacs dumper dispute

Posted Dec 8, 2016 14:54 UTC (Thu) by welinder (guest, #4699) [Link]

Eli showed good technical judgment 20 years ago when I last dealt with him on a
regular basis. (And tried to hire him. Maybe it's time to try again.) If you think you
have found some basic programming thing he doesn't understand, chances are
good you overlooked something.

*My* preferred solution to the dumping issue would be to collect all the elc files
that are currently predumped and concatenate them all into one giant startup
file to-be-loaded, basically, by temacs. That's not currently fast enough, but
lexing and parsing elc files should be possible at hundreds of megabytes per
second.