Why modify the executable? [LWN.net]

Why modify the executable?

Posted Jul 15, 2009 21:04 UTC (Wed) by nix (subscriber, #2304)
In reply to: Why modify the executable? by epa
Parent article: Is pre-linking worth it?

That's inaccurate. prelink modifies the relocations in the executable
itself: it doesn't add a new section describing them.

It *does* add a section describing the 'conflicts' that it *cannot* ---
there are two per C++ virtual method table, for instance --- and a section
allowing it to undo its own work, but there is no new section describing
relocations. The glory of relink is that to a large extent all the dynamic
loader needed to do to work with it was to know when to get out of the
way.

And it has a huge positive benefit for KDE, so large that a kludge
(kdeinit) had to be implemented specifically to prevent a massive slowdown
when prelink wasn't active (also because without kdeinit you'd have
to execute a new program to handle *every single URL* that was loaded
while loading a webpage, for instance). That's because KDE programs link
against a lot of C++ shared libraries, and even with .gnu.hash
ameliorating the 'long C++ symbols take ages to relocate' problem, the
libraries are so large that relocation takes ages (even with DT_BIND_NOW
off: some of the relocations have to be processed immediately and can't be
handled lazily).

An example: on my KDE 3.5.10 system here, kio_http (a shared library
itself) and dependent libraries contain a total of 48929 symbols with an
average length of 31.9. (Taking C++ symbols only, the average symbol
length is 35.4: most of the symbols are C++). You can expect a
nonprelinked KDE program to use a meg or so of extra dirty nonshareable
memory just for the relocations (I haven't looked at this figure for some
time: it'll be worse on 64-bit boxes but of course they normally have more
memory anyway).

That is *not* small. prelink really does help with monsters like this,
even with DT_GNU_HASH to help speed up relocation when it must happen.

And programs are only going to get bigger.

to post comments

Why modify the executable?

Posted Jul 15, 2009 22:48 UTC (Wed) by pynm0001 (guest, #18379) [Link] (1 responses)

A couple of small quibbles: Although KDE downloads webpages through a separate process, it does not necessarily have to fork one for every link visited. Existing kio_http processes (procs, not shared libs) are retained for some amount of time after they've been forked in case they are needed to be reused.

kdeinit existed before prelink, although essentially to fix the same problem.

Why modify the executable?

Posted Jul 15, 2009 23:15 UTC (Wed) by nix (subscriber, #2304) [Link]

Ooo, kio_http is in effect thread-pooled?

[digdig]

So it is! I never knew that. Nifty (also bleeding obvious in hindsight,
given that I can see some hanging around even now, when I'm not actively
downloading any web pages. Ah well, the glory of the Internet is that one
can display one's ignorance in front of thousands).

Are those C++ symbols all external?

Posted Jul 16, 2009 10:00 UTC (Thu) by sdalley (subscriber, #18550) [Link] (5 responses)

nix, is your KDE 3.5.10 built with a recent compiler? All those 48929 C++ library symbols - do they *have* to be externally visible?? Or are most of them object-internal/mangled-static symbols which shouldn't need to be involved in the linking at all? The -fvisibility compiler option is supposedly used (gcc 4.1 and later) to restrict the visibility of internal symbols, are there 48929 externals in spite of that?? That's a *huge* interface surface to document and test if so.

Are those C++ symbols all external?

Posted Jul 16, 2009 17:47 UTC (Thu) by nix (subscriber, #2304) [Link] (4 responses)

That's in spite of using hidden visibility, yes. (The whole thing was
built with GCC 4.3.2.)

(Actually I'm not sure whether hidden visibility ever really worked with
KDE 3.x. I'll look again at a KDE4 installation as soon as I unbreak mine
far enough that a decent subset of the libraries are actually there and
have the symbols they're meant to have.)

Are those C++ symbols all external?

Posted Jul 19, 2009 7:36 UTC (Sun) by dirtyepic (guest, #30178) [Link] (3 responses)

it supposedly does work with KDE 3, or at least I remember us having to disable it repeatedly as support for hidden visibility matured over GCC releases. these days though it works pretty smoothly. note, however, that it's disabled by default. you have to pass --enable-gcc-hidden-visibility to configure at build time.

Are those C++ symbols all external?

Posted Jul 19, 2009 11:25 UTC (Sun) by nix (subscriber, #2304) [Link] (2 responses)

Last time I did that it refused, telling me Qt (3.3.8 with security fixes)
didn't have sufficient support. I think it needs distro patches to Qt...

Are those C++ symbols all external?

Posted Jul 20, 2009 4:39 UTC (Mon) by dirtyepic (guest, #30178) [Link] (1 responses)

I think you're right. I never noticed that before. I find it odd that a feature in KDE would require external patches to Qt, but I guess that's the kind of situation they were in at the time.

Are those C++ symbols all external?

Posted Jul 20, 2009 6:50 UTC (Mon) by nix (subscriber, #2304) [Link]

Getting hidden visibility to work with C++ was tricky, as I recall. You
certainly have to appropriately mark ancestors of classes that are being
so marked to get any real reduction in symbol count, which means Qt does
need marking.

Why modify the executable?

Posted Jul 20, 2009 18:14 UTC (Mon) by oak (guest, #2786) [Link] (1 responses)

> on my KDE 3.5.10 system here, kio_http (a shared library itself) and
dependent libraries contain a total of 48929 symbols with an average
length of 31.9. (...). You can expect a nonprelinked KDE program to use a
meg or so of extra dirty nonshareable memory just for the relocations

I would expect that many symbols to take several megs nonshareable memory
for the relocations per process...?

Why modify the executable?

Posted Jul 20, 2009 21:55 UTC (Mon) by nix (subscriber, #2304) [Link]

Stop expecting me to do difficult things like multiplying numbers by a
fixed constant. :) And yes, you're right, it is several megs.