Fedora opens up to bundling

Posted Oct 14, 2015 15:29 UTC (Wed) by raven667 (subscriber, #5198)
In reply to: Fedora opens up to bundling by josh
Parent article: Fedora opens up to bundling

This is exactly the problem that drives software developers, who aren't subordinate to the distro, to bundle their dependancies, because it's the only way they can get a stable underlying environment to develop and test on, that really doesn't have anything to do with whether the source is open or not. Either the developer spends their time testing across the various bits of libraries, compiler options, etc. that make the distros different, or each distro spends some non-zero amount of effort porting and testing the software against their unique snowflake of libraries and options, that then has to be re-done for every major upgrade, but this is a non-trivial amount of work in aggregate. The developer can sidestep a bunch of this by bundling libraries, even more by containerizing and bundling an entire OS runtime that they can test and configure.

If distros want developers to unbundle their dependancies then the distros need to start making some promises about compatibility, for 5-7 years, of the libraries they want developers to be able to depend on otherwise this doesn't fix the pressure toward bundling and containerizing.

Fedora opens up to bundling

Posted Oct 14, 2015 15:40 UTC (Wed) by hkario (subscriber, #94864) [Link] (23 responses)

There are distributions that provide support and API stability for 10 years - RHEL being one example. Even with Ubuntu LTS you get 5 years.

But developers don't want to program to old API that is in a 2-3 year old distro. "Screw support, I need this newest and greatest feature from this unstable development branch of foobaz library."

You can't have the cake and eat it.

Fedora opens up to bundling

Posted Oct 14, 2015 16:37 UTC (Wed) by raven667 (subscriber, #5198) [Link] (9 responses)

Sure, they support stability by never changing anything, only carefully backported fixes, with limited provision to run applications from different API levels at the same time. Extending support for bundling or containers or just multiple versions of libraries on the same system would be more do-able if there were some sort of agreed-upon target. We could make a hypothetical Linux-ABI-2012 run on the same kernel as Linux-ABI-2016 with the ability to mix and match user applications on the same system, if we had those things defined in any kind of sensible way. The way this is being done now is with containers, which has its own downsides.

Fedora opens up to bundling

Posted Oct 14, 2015 16:49 UTC (Wed) by josh (subscriber, #17465) [Link] (8 responses)

There's a reason the LSB has all but died (https://lwn.net/Articles/658809/): nobody actually seems to care, and nobody actually writes to that ABI.

Fedora opens up to bundling

Posted Oct 14, 2015 17:45 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (7 responses)

That's because LSB took a wrong approach - it tried to standardize too much and achieved too little. And its toolkit is also complicated to use.

Fedora opens up to bundling

Posted Oct 14, 2015 18:23 UTC (Wed) by josh (subscriber, #17465) [Link] (1 responses)

They tried to standardize enough to allow useful applications. Applications often use a huge number of libraries and system functions. Standardizing a half-dozen libraries will not suffice, and POSIX does not suffice (especially for non-command-line apps).

Fedora opens up to bundling

Posted Oct 14, 2015 18:36 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Yes, and it didn't work out. For example, they'd added Qt4 just at the time it went into the EOL lifecycle stage.

Fedora opens up to bundling

Posted Oct 17, 2015 13:32 UTC (Sat) by krake (guest, #55996) [Link] (4 responses)

I don't see it as trying to standardize too much, but I agree that they took a wrong approach.

In my opinion, the mistake was to standardize on what was already shipped, instead of specifying what should be shipped.

Basically catering to the wishes of the distributors instead of the needs of the developers.

Fedora opens up to bundling

Posted Oct 18, 2015 2:41 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> In my opinion, the mistake was to standardize on what was already shipped, instead of specifying what should be shipped.
That's an even deeper mess. Now a committee will have to design API for the future, without feedback from actual developers. This only leads to pain, suffering and the original POSIX specification.

Fedora opens up to bundling

Posted Oct 18, 2015 8:56 UTC (Sun) by krake (guest, #55996) [Link] (1 responses)

Right now the situation is closer to what you describe, i.e. specifying without input from developers.

Instead of providing something that developers would want to develop against, they choose to specify something that distributors are automatically in compliance with.

Of course that is great for marketing, but rather useless for engineering.

Fedora opens up to bundling

Posted Oct 18, 2015 9:13 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

Committees are notoriously bad at listening to developers. Most of the successful standards of the last years simply standardized the existing best practices. C++ is a good example of this.

LSB should have realized that the GUI toolkits in Linux can't be standardized, except at the lowest level (X-server or Wayland). And as for the startup system and configuration, LSB tried to be as flexible as possible to accommodate all distributions, so it ended up less than useful for all of them.

Fedora opens up to bundling

Posted Oct 22, 2015 9:39 UTC (Thu) by Wol (subscriber, #4433) [Link]

> In my opinion, the mistake was to standardize on what was already shipped, instead of specifying what should be shipped.

> Basically catering to the wishes of the distributors instead of the needs of the developers.

Sort of yes ...

They shouldn't have standardised on what was shipped, they should have standardised a way of describing what was required...

I tried to get them to do that - the example I give is I wanted to create an LSB virtual package that would pull in the components required by WordPerfect. That approach would have been great - an app developer just specifies the pre-requisites and leaves it to the distro to ensure they are met.

What's the point of describing what's there, if there's no way of prescribing what's needed?

Cheers,
Wol

Fedora opens up to bundling

Posted Oct 15, 2015 10:19 UTC (Thu) by NAR (subscriber, #1313) [Link] (11 responses)

The (other) problem is difference between distributions. Ubuntu LTS has boost 1.54, SuSE has 1.49, debian jessy has 1.55 - now which version shall I use in my application? I presume RHEL uses a fourth different version, just for the fun of it. Oh, by the way, the latest Fedora uses 1.57 (or 1.58?). Does this provide any value for the application developers or users? Absolutely not.

Of course, there's still the classic problem of "I want old stable version of everything except the browser which I want the latest", but the browser depends on newer libraries, so I have to upgrade everything. Unless the distributions start to support all versions of all libraries during their lifetime, application software will bundle their libraries. As they generally tend to lack manpower, I doubt this will come to pass.

Fedora opens up to bundling

Posted Oct 15, 2015 15:50 UTC (Thu) by josh (subscriber, #17465) [Link] (7 responses)

> The (other) problem is difference between distributions. Ubuntu LTS has boost 1.54, SuSE has 1.49, debian jessy has 1.55 - now which version shall I use in my application?

Assuming you don't have specific feature requirements that necessitate a more recent version, you should use 1.54 on Ubuntu LTS, 1.49 on SuSE, and 1.55 on Debian stable. (Hopefully the C++ ABI will stabilize to the point that if you *really* want to build just one version for all distributions, you can build against the oldest you support and run with any newer version. But when you build as part of a distribution, you use that distribution's package and whatever version it provides.)

Fedora opens up to bundling

Posted Oct 15, 2015 18:14 UTC (Thu) by ms_43 (subscriber, #99293) [Link] (1 responses)

Boost provides no ABI guarantees. The only way you can use boost is to bundle it.

Well, most of boost is header-only anyway, so the bundling is fully automated by cpp.

Fedora opens up to bundling

Posted Oct 15, 2015 20:18 UTC (Thu) by josh (subscriber, #17465) [Link]

> The only way you can use boost is to bundle it.

https://packages.debian.org/libboost-iostreams1.58.0

Fedora opens up to bundling

Posted Oct 16, 2015 13:21 UTC (Fri) by NAR (subscriber, #1313) [Link] (2 responses)

So I have to maintain three different versions for absolutely no gain. And gets really annoying when different versions have different bugs requiring different workarounds...

Fedora opens up to bundling

Posted Oct 16, 2015 16:33 UTC (Fri) by josh (subscriber, #17465) [Link]

No, you maintain zero versions, unless you particularly want to be a distribution package maintainer.

Fedora opens up to bundling

Posted Oct 16, 2015 16:45 UTC (Fri) by bronson (subscriber, #4806) [Link]

And it's even worse: you need to maintain three different build environments in three different distros (at least). Even with virtualization, that can take up quite a few hours per week right there.

Fedora opens up to bundling

Posted Oct 16, 2015 16:27 UTC (Fri) by stevem (subscriber, #1512) [Link] (1 responses)

> Hopefully the C++ ABI will stabilize

*giggle* Sorry, that's just pure comedy...

Fedora opens up to bundling

Posted Oct 16, 2015 16:34 UTC (Fri) by josh (subscriber, #17465) [Link]

See https://isocpp.org/files/papers/n4028.pdf and other standards-committee work on this.

Fedora opens up to bundling

Posted Oct 17, 2015 19:33 UTC (Sat) by lsl (subscriber, #86508) [Link] (2 responses)

> The (other) problem is difference between distributions. Ubuntu LTS has boost 1.54, SuSE has 1.49, debian jessy has 1.55 - now which version shall I use in my application?

None of the above. Use a sane library that offers a proper interface and keeps it stable across releases. Then you can pick the feature set of the oldest version you want to support and it will continue to work on all later versions.

Even if all distributions had the exact same version of boost, that wouldn't solve your problem. When boost version n+1 gets released and included in distributions, your stuff would still break unless someone commits to maintaining old boost versions indefinitely. Distros won't (and can't) do that with an upstream that is as hostile to backwards compatibility as boost.

Fedora opens up to bundling

Posted Oct 21, 2015 16:12 UTC (Wed) by rwmj (subscriber, #5474) [Link] (1 responses)

... is the correct answer. The problem isn't just applications, it's libraries not offering long term stable API guarantees. This is sloppy, lazy programming.

Fedora opens up to bundling

Posted Oct 21, 2015 17:43 UTC (Wed) by pizza (subscriber, #46) [Link]

...except for applications that take libraries, mangle them in incompatible ways, and bundle the results. (Chrome/chromium was the most infamous example of this, though I understand they have considerably cleaned up their act in recent years)

Fedora opens up to bundling

Posted Oct 18, 2015 10:57 UTC (Sun) by paulj (subscriber, #341) [Link]

LTS releases, which just don't change anything of note, are a band-aid over the root of the "problem": few distro upstreams take care to preserve compatibility, and the distros never pushed back against upstreams breaking ABI and API regularly.