Quotes of the week [LWN.net]

Quotes of the week

Posted Aug 31, 2012 9:32 UTC (Fri) by ebassi (subscriber, #54855)
In reply to: Quotes of the week by dlang
Parent article: Quotes of the week

quoting myself from a comment on a different thread

that is definitely not what I was asking, and that's not even an answer in this specific context.

I'm talking about ABI and conformance for third party software, not performance.

given that I wasn't clear: let's assume for a minute that a fairy on night gave us an internal ABI for the core platform (libraries and applications) and an external ABI for the rest of the core platform consumers (i.e. developers); this split allows us to have the core platform moving without breaking third party applications ever: if something breaks in the internal ABI, it will be fixed on the gnome.org repository by the author of the change. mostly like how the kernel works.

now, what are the side effects? we effectively have two libraries, one for the internal and one for the external ABI. do we load twice the amount of shared objects? I mean, apparently Enlightenment convinced everyone that it's fine to have a bajillion DSO, as long as you keep repeating that it's fast (hint: it's really not, and anyone with a minimal understanding of how kernel and linkers work knows this), but I don't think it'll scale. do we compile stable and unstable symbols in the same shared object? that works, but it requires re-linking the entire planet because you really want to bump the SONAME so that applications using the internal ABI can actually be updated; if you bump the SONAME, though, applications linking against the external ABI will also need to be relinked, otherwise they won't be able to find the shared object - look at all the apps linking against boost, and breaking after 12 months because nobody ships the old version of boost they use any more.

so at this point third party apps are in the same situation as they are now, even if they use a supposedly stable ABI. what should they do? dlopen() the shared object? fun times - ask any GL developer how that works out - and it does not scale with a shared library of 5000+ symbols or more. ISVs tried to do that with gtk+ multiple times, it didn't work.

but all of this assumes that the ABI is just a bunch of functions - which is not the case in an object oriented environment like GNOME. if we have two ABIs, where does subclassing a GtkWidget live? the internal ABI, that we can break, or the external one? we cannot really change the size of structures in an ABI-compatible way, so it'll have to be in the internal ABI, for core libraries and apps; though that means that external libraries and applications won't be able to create their own third party widgets unless they use the internal ABI - which is utterly broken, because applications need to define their own widgets, as GTK+ cannot possibly ship all possible widgets needed by all apps for all eternity, and do that even before the apps have been written; and in some cases third party apps are closed source, so the authors cannot put their widgets inside an open source toolkit. the whole "non derivative work" status of some kernel modules is still pretty fuzzy, I don't think people want that fuzziness to extend to user space as well.

plus, GNOME is not just C. we do have another ABI in the form of the introspected data that is used to generate bindings for other languages, like Python, JavaScript, and Perl. that is a third ABI that would have to be split in two, and it would require jumping through various hoops, some of which are on fire.

it's always easy to assume that people outside of your restricted area of interest are assholes that do not understand the problem space, especially if you spent about 15 minutes to think about it. I can assure you: we people working on the GNOME core platform discuss this shit all the time. there is simply no good solution, or good enough solution yet.

after reading the comments in Miguel's blog post I have to say, though, that it's always cute how kernel developers think they can solve the user space problems using their limited perspective, like they are the second coming of Jesus Christ; it does take its toll, though, given that we have to explain all this basic software engineering and release management stuff all the time. you'd think they'd get it.

(Log in to post comments)

Quotes of the week

Posted Aug 31, 2012 13:55 UTC (Fri) by khim (subscriber, #9252) [Link]

I'm talking about ABI and conformance for third party software, not performance.

What's your point? If that's a simple whine that creation of stable ABI is hard then I agree, it is hard. It's also necessary.

Do we compile stable and unstable symbols in the same shared object?

The same, of course. What's the point of splitting them?

That works, but it requires re-linking the entire planet because you really want to bump the SONAME so that applications using the internal ABI can actually be updated.

Absolutely not. ELF Symbol Versioning was proposed as a solution for this problem decade ago. Other solutions are possible, too (it's hard to use C++ with ELF Symbol Versioning, but there are it's own solution).

You provide library with both internal and external symbols on the live OS and you provide shims with just public symbols for the external developers. This is how Windows does that, this is how MacOS does that, this is how LSB does that if you want an example closer to the Linux!

if you bump the SONAME, though, applications linking against the external ABI will also need to be relinked, otherwise they won't be able to find the shared object - look at all the apps linking against boost, and breaking after 12 months because nobody ships the old version of boost they use any more.

Well, the problem is obviously not boost. The problem is the stupid idea to include it in a "system code" without accepting the support burden. If you include library like boost in a system core then yes, you need to ship bazillion versions of the same library in a said code. It's a fee you pay for compatibility.

Sane libraries (such as GTK+ or GLibC) bump versions very rarely which reduces said price.

so at this point third party apps are in the same situation as they are now, even if they use a supposedly stable ABI.

That's not a stable ABI. That's a perverse joke. Stable ABI means one simple thing: I can take binary compiled 10 years (or 5 years if 10 years is too much for you), start it - and it'll work. This is an ideal, 100% stable ABI is not really possible (even in Windows some programs stop working). But this is what you should strive for. Is it hard? Oh yes, there are countless stories about silly application developers and insane solution Microsoft comes with (for example programs called install.exe and setup.exe are handled in slightly different fashion than whatsisname.exe). But this is what OS (any OS) is supposed to provide.

it's always easy to assume that people outside of your restricted area of interest are assholes that do not understand the problem space, especially if you spent about 15 minutes to think about it. I can assure you: we people working on the GNOME core platform discuss this shit all the time. there is simply no good solution, or good enough solution yet.

Are you sure you are "discuss this shit" with a goal to find a solution and not with a goal to find an excuse? If I understand correctly you are discussing that "unsolvable problem" for a decade while other people come and solve it and then do it again and again.

KDE was a good example till they decided to drop everything and create a new incompatible version. What was fine for the library (developers can always choose to use one version or another) is not fine for an OS: it must support old interfaces for a long time (years) till most applications (even obscure ones) will catch up to the new ABI. My favorite example is FCB: introduced in CP/M back in 1973, ported to MS-DOS in 1981, obsoleted in 1983 yet supported till Windows 2000. And even then people complained about their beloved WordStart!

And it's not impossible to provide stable ABI even if you do radical redesign. Think MacOS: it provided stable ABI in all transitions. From m68k to PPC to x86-64, from MacOS classic to MacOS X. Every time you had the ability to run old programs. Yes, Apple is not Windows, it drops support for older ABIs regularly. But it only does that when they have persuaded most ISVs (at least the major ones) to switch to a new ABI and when people have mostly upgraded their applications to a new version.

When other companies do the same idiotic radical ABI breakage GNOME and other Linux desktop are doing they suffer the same fate: witness Windows Phone 7.