Dependency bundling
Dependency bundling
Posted Dec 17, 2021 8:13 UTC (Fri) by joib (subscriber, #8541)In reply to: Dependency bundling by ras
Parent article: Lessons from Log4j
Perhaps distros could engage in some introspection why developers have abandoned relying on the distro packages to the extent they have, and what could distros do about it? Bonus points if you can do it without the kneejerk CADT response.
Posted Dec 17, 2021 10:04 UTC (Fri)
by LtWorf (subscriber, #124958)
[Link] (3 responses)
Nobody will follow best practices if they are not enforced.
Of course product management is happier to bundle stuff, because it leads to being faster. And developers are fine with that because they won't need to learn how to package software.
Everyone is happy until disaster happens :D
Posted Dec 18, 2021 2:30 UTC (Sat)
by ssmith32 (subscriber, #72404)
[Link] (1 responses)
People have very different ideas of best practice..
Posted Dec 18, 2021 16:30 UTC (Sat)
by LtWorf (subscriber, #124958)
[Link]
On the other hand version pinning makes sure that security fixes will never reach your product.
Posted Dec 18, 2021 11:02 UTC (Sat)
by Wol (subscriber, #4433)
[Link]
(1) Because "best practice" often isn't.
That's why greybeards do it and newbies don't ...
Cheers,
Posted Dec 17, 2021 20:40 UTC (Fri)
by khim (subscriber, #9252)
[Link] (9 responses)
Even if is the reason for that phenomemon? Because it's the only way to create cross-distro and cross-version binary. And that's what you want both if you want to give you program to the user (most of which are not progammers and don't know how to compile anything) or if you want to use program in-house and retain the ability to upgrade OS without hassle (attempt to rely on distro-provided libraries would lead to pain with installation on upgraded OS because certain versions of certain libraries would become unavailable). An SDK. Some way to build binary which you may build once and use forever. Or, if not forever then at least for 5-10 years. That's what all OSes are providing, just not Linux distributions. That's the bare minimum. If you want to make sure developers wouldn't try to bundle libraries which are not in your base SDK then then would need some cross-distro and cross-version way to deliver other libraries. That's even harder, I don't know any OS which managed to pull that off. Flakpak is trying, AFAIK.
Posted Dec 18, 2021 11:31 UTC (Sat)
by joib (subscriber, #8541)
[Link] (1 responses)
Yes. Because insulting developers will only alienate developers further and ensures that whatever good points distros might have about maintainability and dependency management will fall on deaf ears.
> An SDK.
> Flakpak is trying, AFAIK.
Yes, something like that. Flatpak is probably the best shot in the desktop space.
In the server world, I don't know. Developers using "modern" languages really love things like cargo+crates.io/NPM/whatever, and for good reasons. The challenge is how to integrate those models with some trusted third party (call it a "distro" or whatever), that would ensure long-term maintenance and security updates for some particular versions of particular packages. Oh, and some kind of "apt-get upgrade" type mechanism to semi-automatically rebuild applications with bundled dependencies due to security updates in the dependencies.
Posted Dec 18, 2021 23:52 UTC (Sat)
by NYKevin (subscriber, #129325)
[Link]
For $DEITY's sake, even Steve Ballmer had this figured out. Remember the "Developers, developers, developers!" line? You could have the greatest operating system in the world, but it doesn't matter if nobody wants to write code for it.
Posted Dec 19, 2021 0:43 UTC (Sun)
by NYKevin (subscriber, #129325)
[Link] (6 responses)
In the proprietary world, this is a solved problem. It's called "You can't bundle it or else our lawyers will sue your pants off. Instead, every end user must download the package from upstream, which is installed in a single standard location, and if your OS/app/whatever doesn't like that standard location, then that's your problem."
Before anyone asks: New versions are handled as if they were entirely unrelated packages. You can easily end up with dozens of these "Microsoft C++ Redistributable" nonsense packages on a single Windows box.
Posted Dec 19, 2021 1:11 UTC (Sun)
by khim (subscriber, #9252)
[Link] (3 responses)
Except, of course, these runtimes not only can be bundled, but they were designed to be bundled from very early days. And yes, they, sometimes, needed crazy tricks to support these bundled runtimes. Yet you always had an option to bundle them and you still have that option even today. They only tried to push that approach after taking 90%+ of desktop. After they achieved, essentially, a monopoly. And even then the end result was near disaster: that's how they lost the title of leading desktop platform platform to the web (Ironically enough, after killing Netscape). And web, the platform that won the title of most popular platform on desktop, for now, is all about bundling dependencies. GNU/Linux never had a monopoly, yet it tried to put much harsher requirements on developers. This flew like a lead balloon: most apps today they are developed for web or Windows and macOS, only tiny percentage supports Linux.
Posted Dec 19, 2021 2:57 UTC (Sun)
by NYKevin (subscriber, #129325)
[Link] (2 responses)
Sometimes, there isn't even a reasonable way to determine whether the thing is already installed or not, so you end up doing extra-crazy things like re-running the same installer over and over again (see https://help.steampowered.com/en/faqs/view/2BED-4784-8C0A...).
Posted Dec 19, 2021 14:19 UTC (Sun)
by khim (subscriber, #9252)
[Link] (1 responses)
That was always an option, not the requirement. That's why, after you wrote that, I went and verified, that /MT is where it was always been. Even with just released Visual Studio 2022. Ah. Thanks for providing that link. Now, please go read what's written there yourself. Yes, with DirectX it's done like that. And, later, the trick was repeated with .NET framwork. But there are extremely important difference between what was done there and in Linux world. Microsoft decided from the very beginning that there would be one DirectX and one .NET runtime (later they added few more, but the original ones still are all supported). And they ensured that programs built for DirectX 1.0 (released in 1995, remember?) still would work today (there are some bugs which may prevent it, but there were no on-purpose breakage since 1995, for quarter-century). And when they had that promise they started working on legal enforcements. And yes that combination of technical and legal solutions works. What Linux libraries can you name which are supporting similar technical promises? GLibC? Well, congrats: even most super-duper-bundle-savvy apps very rarely bring their own version of GlibC. Even if, technically and legally, they can. The important part of the solution (outlined in the link you have shared, ironically enough) was not done for GLibC (there are no way to bring your own version of GlibC and install it), but glacial speed of development worked as an adequate substitute: GlibC is so low-level and changes so slowly that using 5 years old version is not too painful. Yeah, that's rare success. Of GlibC developers, not distro makers, though. Everyone else liked to play CADT games which made the desire of distro-makers to have just one copy of each library unrealistic: where there are no compatibility between versions there would be bundling… it's as simple as that. Yet distributors tried to fit that square peg into a round hole for decades… with very little success. I'm also guilty: I tried to help with creation of one local distribution, years ago, got fed up with all these incompatibilities (when we tried to somehow invent a way to run apps from RedHat on our distro) and switched back to Windows.
Posted Dec 20, 2021 18:36 UTC (Mon)
by NYKevin (subscriber, #129325)
[Link]
Side note: As someone who was actually diagnosed with AD(H)D, I really wish people would stop using my condition as a pejorative.
Or claiming that it doesn't exist, for that matter.
Posted Dec 19, 2021 13:36 UTC (Sun)
by smurf (subscriber, #17840)
[Link] (1 responses)
The Distribution approach ("there is exactly one copy of FooLib on the system which everybody uses, it gets security fixes only; if you need a newer copy you get to wait for the next distro release") may not be for everybody but at least it solves *that* problem.
Posted Dec 19, 2021 13:49 UTC (Sun)
by khim (subscriber, #9252)
[Link]
Of course there are ways to prevent that! Windows never had that problem in the first place. And Android implements a way solve it. Not sure what macOS is doing, but hope it's either avoided the problem in the first place (like Windows) or solved it (like Android). True. If your sausage is raw and under cooked then go and burn the whole house down. That sure would solve that issue: most likely you would have no sausage to worry about, but if it would, by some freak chance, survive then it would be well done.
Posted Dec 18, 2021 1:01 UTC (Sat)
by pabs (subscriber, #43278)
[Link]
Dependency bundling
Dependency bundling
Dependency bundling
Dependency bundling
(2) Because if you're young (and "foolish") you don't understand the concept.
(3) Because if you haven't been burnt you don't see the point.
And, rather importantly
(4) if you're a manager it's someone else's problem...
Wol
> Bonus points if you can do it without the kneejerk CADT response.
Dependency bundling
Dependency bundling
Dependency bundling
Dependency bundling
Dependency bundling
Dependency bundling
> IMHO that doesn't count as "bundling" because, in most cases, at least on modern systems, the thing that is bundled is a binary blob self-extracting installer you got from upstream, and you are in no way allowed to just install random DLLs into your Program Files directory.
Dependency bundling
Dependency bundling
Dependency bundling
> Yeah, and then you have a library linked against FooLib 3.2, another library using FooLib 3.3, and no way to prevent these two from stepping on each others' toes when you try using them in the same application.
Dependency bundling
Dependency bundling