You are correct [LWN.net]

You are correct

Posted Feb 1, 2008 0:00 UTC (Fri) by jd (guest, #26381)
In reply to: Why companies don't support Debian (LinuxWatch) by NightMonkey
Parent article: Why companies don't support Debian (LinuxWatch)

Gentoo's only crisis is that at the moment, it has no charter. Debian's only real problem at the moment is the same one all package-based distros face - there are too many stale packages, yet also far too few packages in total.

Developers are massively out-pacing the ability of packagers to package what is produced, way too many package permutations are possible but mutually exclusive, and there's way too much free software out there for a single vendor to coordinate, yet uncoordinated repositories are what caused the RPM hell too many of us have been burned with. Even "unified" repositories aren't immune. Ubuntu's packages aren't compiled against a uniform environment, creating some fascinating dependency conflicts.

I have considered starting a repository of otherwise-unpackaged software, to in an effort to solve some of this, but the bottom line is that this is an overwhelming problem. I have bookmarked as many unpackaged free software products that show signs of having plenty of users as Debian, *buntu and Fedora have actually provided in repositories. Their efforts have taken thousands - if not tens of thousands - of man-hours to accomplish, verify, test and supply. I could never keep up with any meaningful portion - at least whilst keeping a job and a roof over my head - the best I could do is some insignificant fraction, which is what others already do and which is the cause of incompatibility problems.

In other words, repeating other people's mistakes would at least be doing something, it would merely be doing the wrong something.

If corporate sponsorship was to enter into the distro picture, I'd say that it would need to enter at the weakest points: testing and packaging. These are the areas distros have never done well in and therefore really should consider handing off to wage slaves. If they don't want to, then they should find other solutions. What they currently do is broken. Fix it or replace it, just stop ignoring it.

(Log in to post comments)

You are correct

Posted Feb 1, 2008 0:38 UTC (Fri) by N0NB (subscriber, #3407) [Link]

Too much Free Software???  I think I understand your point that the Free Software community is
generating code faster than the distributions can keep up.  That is a fair assessment.

As I see it, Debian is drifting toward putting the best of the best into Main and putting less
emphasis on trying to package every piece of Free Software on the planet for the distribution.
This is a sensible approach but it does lead to the issue that distributions were created to
address, that of users having to build their own packages from source.

Ideally, Free Software projects would create their own Debian packages, but this becomes a
problem as well.  At some point a package may depend on something not available when Stable
was released and Unstable at some points moves fast enough that a package created today is
outdated next week, so it's unrealistic for Free Software projects to expend the man-hours to
sort this all out.

Perhaps this is an opportunity for an enterprising person or company--build unofficial Debian
packages of Free Software not in or not up-to-date in Debian and make them available for a
small fee.

Exactly.

Posted Feb 1, 2008 7:31 UTC (Fri) by jd (guest, #26381) [Link]

It is unreasonable for Free Software projects to put in that level of extra work, but it would be entirely reasonable for corporations (who rely on getting bugfixes and security fixes as fast as possible) to invest in precisely this sort of work. In fact, it's insane for them not to, because they suffer more than anyone else. An hour downtime costs far more than an hour of QA and patch development.

One relatively cheap solution (for a corporation) is to produce a huge server farm, where multiple versions of packages are built, dependent on different valid parent packages. Package managers would need to be somewhat smarter, because they'd need to search multiple alternative versions to find the optimum permutation of valid packages, which may mean swapping an existing package for an alternative of the same version but with subtly different dependencies.

There are plenty of other options. What's really not an option (in my opinion) is to have distributions become overly specialized to remain fully working, or highly generalized (good) but dependent on people not staying up-to-date (bad). Users are likely to end up having to compile some things for themselves, the aim (I believe) should be to keep this to a minimum and to make it as painless as possible when it does happen.

Exactly.

Posted Feb 8, 2008 2:09 UTC (Fri) by vonbrand (subscriber, #4458) [Link]

I'm sorry, but "the corporations" that you want to pick up the slack do have a much narrower view than "the community". If said community doesn't want to pay for something, be it testing, packaging, whatever, it won't get done for the all-encompassing interest of something like the set of Debian users. And by "pay" I mean not necessarily money, but in effort.

The "corporations"

Posted Feb 8, 2008 19:25 UTC (Fri) by jd (guest, #26381) [Link]

...want to sell their products to Linux business users. They are there to make money, after all. Linux business users get the same Linux software everyone else gets, more-or-less. If that software is unreliable, business users won't use it, which means Linux products won't sell.

Why sell to Linux users in the first place? Too many variants of Unix, too many of which only have a subset of the features needed or wanted, too much competition from Microsoft and generally not much security. A unified baseline that has Linux has a key platform has far better odds of long-term survival. Provided it can be brought to some provable (rather than anecdotal) quality level.

But to do that, money must be spent on Linux QA. You've got to make people want Linux, want it so bad that alternatives aren't worthy of consideration. That won't happen until there has been some serious QA.

What if no QA happens at all? Then businesses will regard Linux as something vendors aren't interested in, and those businesses will go elsewhere, costing those vendors potential sales. Sure, it would be nice if the Debian community did more QA work, but really QA is a full-time job and most Debian deveopers have two or three of those already. It's a dirty job, but someone has to do it.

The "corporations"

Posted Feb 8, 2008 21:38 UTC (Fri) by vonbrand (subscriber, #4458) [Link]

Who says that the people selecting packages and building "enterprise" distributions don't do QA? They have to restrict the breadth of software offered if they want to keep some sanity in all. Yes, that means that it is very probable that "my favorite package" isn't in the supported set.

You are correct

Posted Feb 1, 2008 3:57 UTC (Fri) by salimma (subscriber, #34460) [Link]

"Ubuntu's packages aren't compiled against a uniform environment, creating some fascinating
dependency conflicts."

Dependency conflicts are unavoidable in some cases, due to package churn. To avoid it you'd
have to state that the packages that ship with the distribution is not allowed to break API
during the lifecycle of that distribution.

Yes.

Posted Feb 1, 2008 7:13 UTC (Fri) by jd (guest, #26381) [Link]

For 100% avoidance, yes, you are right. For a superior level of avoidance, it should be sufficient for the following conditions to be met:

Package configuration files would need to be generated, not static. One way to do this would be to:
1. Examine non-local dependency information in the makefiles
2. Examine linkage information in the compiled binaries
3. Examine build configuration files for libraries and utilities not referenced in the first two steps
4. Reverse lookup what external packages are installed that supply the requisite headers, libraries and utilities
Dependency trees would need to be rebuilt. One way to do this would be to:
1. Regard each dependency as a leaf node of an n-ary tree. Recursively apply this step until all root nodes for all trees have been obtained.
2. If a package has changed OR any package further up the tree has been recompiled, the package is recompiled.
3. If a package has not been recompiled, the only important dependent package is the one already known on that dependency tree. The previous step is then applied to that package alone.
4. If a package has been recompiled, then all packages which depend on it must be identified and recompiled. Since a package is only skipped if a dependency has not been recompiled, this step alone is recursively applied until there are no further dependencies on anything in the tree that has been built.

This obviously doesn't work if there is a break in an API. Anything past a breakage point cannot be refreshed in this way. Not if you use LSB and dynamic linking. You could do it if you had version isolation (a working library can always be present) -or- if the build machine used version isolation and static linking on old libraries to hide legacy details.

There is, of course, another option - controlled slippage. Have a declared distro standard that libraries and utilities should provide a means of emulating old APIs some sensible number of generations back, and that packages have some appropriate time to be fixed or have a distro patch that brings them up to the active API. This doesn't constrain anyone in practice. If anything, it constrains fewer people, as it eliminates many constraints on the distro maintainers and users.

These are heavyweight solutions, yes, which means that you'd need some serious manpower (and compute power) on some of the most tedious areas of packaging. There simply aren't the resources to do anything like this without some corporate involvement, but since these are the sorts of things corporations do well (whereas coding isn't) and like (applications working well together), this isn't too much of a problem. It doesn't impact developers or their freedom, because that's not where the problem lies, but does boost areas of usability that are currently a pain.

unsupported.debian.net

Posted Feb 2, 2008 2:21 UTC (Sat) by pabs (subscriber, #43278) [Link]

I got pissed off with Debian QA people chucking out packages with "no users" so I'm
considering starting unsupported.debian.net. Recently there was a post about automatic
discovery of free software and creation of packages, which would tie in with it well.

If you are interested in reading the writeup of my idea, please mail me, address on
http://wiki.debian.org/PaulWise