libstdc++ licensing

Posted May 7, 2013 20:56 UTC (Tue) by Wol (subscriber, #4433)
In reply to: libstdc++ licensing by khim
Parent article: Stallman: The W3C's Soul at Stake

Actually, while I don't know too much about it, I think it always makes sense to distribute libstdc++ statically, if you're not distributing it as part of an OS.

How doees linux cope with the same library compiled with different versions of the compiler? I know we use .so numbering to avoid dll hell, but I believe C++ neatly gets round that by changing the ABI based on the compiler used ... :-)

Cheers,
Wol

to post comments

libstdc++ licensing

Posted May 8, 2013 2:17 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (3 responses)

> How doees linux cope with the same library compiled with different versions of the compiler?

One thing I've encountered is that in Linux (and likely other ELF implementations, though I've not tested beyond Linux) is that the symbol table is global. Given the following setup:

> A -> C.1
> B -> C.2
> D -> C.1

Where A and B link to 2 different libraries C.1 and C.2 which share symbols (newer version of some dependency, unfortunate collisions, whatever).

If A is loaded first followed by B, what happens at runtime is:

> A -> C.1
> B -> C.1

for the symbols common between C.1 and C.2. What I wish that would happen is that the linker would see that A linked C.1 and therefore *only* look for symbols in C.1.

One example we keep hitting is that some external, third party program uses a dependency that we use (e.g., Matlab and Boost). Matlab uses some old version of Boost (1.3x) and our project requires 1.4x. Since we load the Matlab with dlopen, Matlab gets very unhappy calling the newer Boost symbols with the same name. What we have to do is "#define boost boost_custom_version" in boost/config/user.hpp so that the symbols don't collide for our project. The better solution (that would work today) would be if Matlab used "#define boost boost_matlab_VERSION" (since I doubt MathWorks is going to let anyone else compile Matlab anytime soon) instead of forcing everyone else to get out of its way.

The ideal solution (which I believe Windows uses, but I'm not 100% sure) is the "only look up in linked libraries". RTLD_LOCAL isn't a solution because then D fails to load because it can't find C.1 symbols because A loaded it transitively as RTLD_LAZY and locked all the other libraries from it.

I guess this could interfere with LD_PRELOAD from working as intended, but maybe it could be solved by injecting it into the "allowed libraries for symbol resolution table" for all libraries in the process.

> but I believe C++ neatly gets round that by changing the ABI based on the compiler used ... :-)

The inline namespace trick introduced in C++11 would help tremendously, but that's years down the line for common usage. If multiple symbol versioning could be done with C++ class methods without touching headers[1], it could be done today (maybe it is, but I've been unable to find docs).

[1]Users shouldn't care that there are 3 versions of foo::bar floating around; they should just get the latest version of the method, but the old versions should still be in the source file. Versioning the entire class results in quite a bit of code duplication.

libstdc++ licensing

Posted May 8, 2013 8:46 UTC (Wed) by khim (subscriber, #9252) [Link] (1 responses)

The ideal solution (which I believe Windows uses, but I'm not 100% sure) is the "only look up in linked libraries".

That's not an ideal solution: this means you can not have few different implementations of the same ABI (think gnutls vs openssl) and switch them at runtime. And Windows still uses only filename to resolve collisions thus problems are still possible.

What you want to do in Linux is to use different version strings for different versions of libraries (especially for the libraries like boost which are changing ABI often), but sadly only few standard libraries do that - everyone else pray to "our distribution makers will just rebuild the whole world to keep everything compatible". Kind of ironic to see that once upon time FSF promoted technologies which are good mostly for "nonfree" programs.

The inline namespace trick introduced in C++11 would help tremendously, but that's years down the line for common usage.

… if you forget the fact that inline namespaces are just a minor improvement over ages-old "using namespace nested;". Sure, "using namespace nested" is problematic if you want to do, e.g., partial specialization of templates but these cases are extremely rare: I've built Gentoo system with thousand of packages after building gcc with --enable-symvers=gnu-versioned-namespace and only had couple of problems in places where things were broken by design (where people forward-declared things in std:: namespace instead of including proper header).

IOW: there are more then enough ways to solve compatibility issues in Linux, but to actually do that you need to think ahead and this is extremely unfashionable nowadays. It's much easier to complain on forums about lack of a "silver bullet" (when new "silver bullet" arrives people quickly find out that it does not solve the problem of careless design too and continue complains with abandon).

libstdc++ licensing

Posted May 8, 2013 9:16 UTC (Wed) by khim (subscriber, #9252) [Link]

Sure, "using namespace nested" is problematic if you want to do, e.g., partial specialization of templates but these cases are extremely rare: I've built Gentoo system with thousand of packages after building gcc with --enable-symvers=gnu-versioned-namespace and only had couple of problems in places where things were broken by design (where people forward-declared things in std:: namespace instead of including proper header).

Oh, sorry, I take my words back: in reality it all was based on "inline namespaces", of course - my memory is just fuzzy. As this proposal claims "inline namespaces" functionality was added (under the "using namespace nested __attribute__ ((strong))" guise) more then eight years ago and is now supported on all Linux distributions (including such slow-moving as Debian or RHEL). I've used them in my work about seven years ago. Plain "using namespace nested" approach is even older, of course, but it's less compatible.

libstdc++ licensing

Posted May 10, 2013 15:08 UTC (Fri) by nix (subscriber, #2304) [Link]

Every library glibc loads can have a unique order in which it searches libraries for symbol (it calls these 'search scopes'). By default, for compatibility with projects that started out using .a libraries, all libraries use the same search scope, and symbols named in earlier-loaded libraries interpose the same symbols used later, even in calls within the library. But declaring symbols hidden avoids this for those symbols; linking with -Bsymbolic or -Bsymbolic-functions avoids it for all symbols in a given library for references within that library; using -Bgroup causes all lookups from within that library to be done only within that library and libraries within the group it is part of (defined via --start-group and --end-group); using dlmopen() causes the dlmopen()ed library to get an entire new namespace sharing no symbols with the old...

... there are lots of ways to get different namespaces. Some are poorly supported (e.g. dlmopen()/dlclose() pairs fail after 30-odd attempts on glibc as currently constituted, and there is a static limit of 16 namespaces at present), but the interfaces are there and it sort of works.

libstdc++ licensing

Posted May 8, 2013 8:19 UTC (Wed) by khim (subscriber, #9252) [Link]

Actually, while I don't know too much about it, I think it always makes sense to distribute libstdc++ statically, if you're not distributing it as part of an OS.

Have you actually looked on the original comment? I'm explicitly talk about distribution of libstdc++ as part of an OS.

How doees linux cope with the same library compiled with different versions of the compiler? I know we use .so numbering to avoid dll hell, but I believe C++ neatly gets round that by changing the ABI based on the compiler used ... :-)

ABI is open on Linux (it's not like some other OSes) which means you can keep your compilers compatible. E.g. GCC has few versions of name mangling (to cover cases which are impossible to cover in older versions of it), but it explicitly only changes the default when major version of libstdc++ changes.