topic: is Debian's packaging good or bad [LWN.net]

topic: is Debian's packaging good or bad

Posted Feb 15, 2010 23:49 UTC (Mon) by drag (subscriber, #31333)
In reply to: topic: is Debian's packaging good or bad by coriordan
Parent article: Moblin and Maemo to merge

To drag: "lack of scalability ... huge bottleneck ... currently showing strains" - I remember people saying that ten years ago, when Debian had only a fifth of the packages it has now. How much more scaling and lasting will Debian have to show you? :-)

Until Debian gets a tiny fraction of the level of applications that something like Windows supports then I'll be convinced. Right now Debian is at the miniscule level.

Debian was having scalability fractions before and they continue to do so. They never have changed anything to address the problems they were facing. So far just today I was looking around for multimedia applications and found at least 3 open source Linux applications that I wanted to try out and were not provided by Debian.

Sure I could compile and install them, but that would take quite literally hours of my time. It is not worth it so I am not going to bother.

In fact it's a cronic problem I face every time I want to do something in Linux.

I want to play around with Python bindings for Ogre or other gaming engines. Crystalspace integration with Blender. I want to play around with this or try out that. And you know what? At every turn I am forced to figure out dozens of dependencies... many of which conflict with what I have installed or already and I have to download and compile from scratch. Many times I am forced to create chroot trees to try to build and run software in Linux and rarely I am able to figure it out on the first, or even the third try.

I have to read wikis, google around on mailing lists, and occasionally give up all hope and seek out a IRC channel for help installing software in Linux. Why? Because every project is different and compiling and installing software is usually something that developers work out for themselves and is not really intended for end users to fart around with. I regularly spend hours trying to get something installed... and I having problems prior to even knowing if the software will solve my problems or even be remotely useable!

And you know what? For things like Ogre Python bindings... which multiple times I've spent entire DAYS of my life trying to get working.... Had lovely little windows EXE files that I could of been up and running in MINUTES.

That is right. For the majority of people installing software, especially when it comes to any sort of graphics or gaming or anything of that nature, are much better off using Windows to install and play around with open source software then Debian or Fedora or anybody else.

No Debian installers. No Debian support. No Fedora support. No nothing. No Linux installation support of any kind besides tarballs. And the current crop of Debian developers are overworked and are now seem to regularly start to drop applications that are of no interest to themselves. The length of the process to get to be a official Debian maintainer is YEARS.

Go and look at here: http://www.happypenguin.org/

How many of those are going to get packaged by Debian? How many of those will ever get packaged by anybody?

Sure a lot of them are not very good pieces of software and they wont' be around for very long. However if I was using Windows and they built them for Windows I could have fun playing around with them in very short order. However because I am a Linux user I have to sit and fart around with a project for at least a half a hour to get it running, if I ever get it running.

THAT is why Debian's repository approach does not scale and will never scale. It's _ALREADY_FAILING_. There is a metric crapload of Linux software that will never get any attention, will never make its way into Ubuntu's repository, will never be made easily avialable to end users. It may be good, or it may not be good. The likelihood of anybody finding out about it is pretty damn small.

Debian does a huge amount of work and they do a very good job at what they do. The work they are putting in is critical to the usability that Linux already has. But, especially, when you look at what Fedora, Suse, and everybody else is doing... it's a huge replication of effort that is not really accomplishing anything beyond making installation files for their own specific slice of the user population with very little benefit to anybody else.

(Log in to post comments)

topic: is Debian's packaging good or bad

Posted Feb 16, 2010 0:14 UTC (Tue) by anselm (subscriber, #2796) [Link]

So far just today I was looking around for multimedia applications and found at least 3 open source Linux applications that I wanted to try out and were not provided by Debian. [...] THAT is why Debian's repository approach does not scale and will never scale. It's _ALREADY_FAILING_. There is a metric crapload of Linux software that will never get any attention, will never make its way into Ubuntu's repository, will never be made easily avialable to end users. It may be good, or it may not be good. The likelihood of anybody finding out about it is pretty damn small.

Setting up a Debian repository isn't exactly rocket science. What's to keep you (or indeed anybody) from getting together with a bunch of other interested people and packaging that mythical software that the actual Debian developers are too overworked to package for you, if you think it is getting overlooked? It's not as if one needed to be a Debian developer to be able to run one's own repository. (In fact, one doesn't even need to be a Debian developer to package software for Debian. Uploading it to the Debian servers is what requires a Debian developer.)

Incidentally, speaking of multimedia software for Debian: There is a very nice semi-official repository at www.debian-multimedia.org that does have quite a number of interesting software packages to do with multimedia.

topic: is Debian's packaging good or bad

Posted Feb 16, 2010 0:51 UTC (Tue) by kragil (guest, #34373) [Link]

You didn't get it.

There is just too much software and distros and not enough packagers. It
does not scale to the point where all software is packaged for even one
distro. (Debian comes closest, but some games have copyrighted artwork etc.
and those things generally don't help )

The Linux Foundation really needs to come up with some solution to this
problem. The LSB obviously didn't solve the problem.

Maybe Meego will help in this regard.

topic: is Debian's packaging good or bad

Posted Feb 16, 2010 1:26 UTC (Tue) by coriordan (guest, #7544) [Link]

When thinking about Linux Foundation, remember that although the name might mean something nice to you, it's just a name, and the funding and members of the board comes 85% from of a bunch of proprietary software companies.

In this case, free software has more flexibility than binaries because it can be recompiled to adapt to certain system changes. For the Linux Foundation's members' proprietary software, this benefit is useless, so they'd happily trade it away for crumbs during the setting of standards that help them get the binaries of their proprietary software installed on GNU/Linux systems.

topic: is Debian's packaging good or bad

Posted Feb 17, 2010 4:32 UTC (Wed) by rqosa (guest, #24136) [Link]

> a bunch of proprietary software companies

They're primarily hardware companies rather than software companies. Notice that only 1 out of the 6 "Platinum Members" is not a hardware manufacturer (that one is Oracle).

> standards that help them get the binaries of their proprietary software installed on GNU/Linux systems.

Really? The list of LSB-certified applications includes applications from 4 different companies, and I don't see any of those 4 on the list of Linux Foundation members.

topic: is Debian's packaging good or bad

Posted Feb 17, 2010 21:06 UTC (Wed) by nix (subscriber, #2304) [Link]

Of course, Oracle *is* now a hardware manufacturer.

topic: is Debian's packaging good or bad

Posted Feb 16, 2010 10:16 UTC (Tue) by Klavs (subscriber, #10563) [Link]

> There is just too much software and distros and not enough packagers. It
> does not scale to the point where all software is packaged for even one
> distro. (Debian comes closest, but some games have copyrighted artwork etc.
> and those things generally don't help )

Well - IMHO the situation is much like windows. If I want something to be installable on windows - I need to build an installer in an "exe"-package (or use an existing one).

If I want the software installed on debian or derivatives I would create a .deb package - and I could host it in my local repo - it's just as easy (if not more so) than windows packaging.

Also - one could build an RPM instead (installing under /opt would be recommended) - and that could be used by all (since debian has alien).

Either way -you should - package your software - but you don't have to.
a staticly, compiled tar package will work everywhere - as it would on windows - but no automatic dependency resolving etc. and no easy un-installing.

topic: is Debian's packaging good or bad

Posted Feb 16, 2010 11:07 UTC (Tue) by anselm (subscriber, #2796) [Link]

There is just too much software and distros and not enough packagers. It does not scale to the point where all software is packaged for even one distro.

Maybe not all software actually needs to be packaged?

Debian operates under the assumption that interesting stuff will eventually be packaged. »Interesting« in this context means »interesting to somebody in the position to package it«. For example, I maintain a few packages that probably not a lot of people use, but I do, and so they're in Debian GNU/Linux. If I at some point stop using and maintaining the packages in question, whether they will continue to be part of Debian depends on whether someone else steps up to continue looking after them.

This is an application of the principle that free software works best if it scratches somebody's itch. If some software isn't packaged yet that may mean (a) the place it is supposed to scratch doesn't itch anybody badly enough, or (b) the scratching it does doesn't (yet?) provide adequate pain relief. Often new stuff needs to settle down somewhat before it makes sense to package it. Finally, upstream developers who would like to see their stuff packaged are free to do that themselves (and enlist the Debian project's help in getting the packages into the distribution).

Hence, so as far as Debian is concerned I don't actually think that there is a problem. One might conceivably ask why we need Red Hat's, Novell/SUSE's, and Mandriva's similar-but-separate offerings in the RPM-based distribution area, but then again it is as well to remember that these three are, unlike the Debian project, all commercial competitors who are understandably reluctant to throw their lot in with each other. (That was actually tried a number of years ago -- remember UnitedLinux? -- and didn't get anybody anywhere.)

topic: is Debian's packaging good or bad

Posted Feb 17, 2010 2:48 UTC (Wed) by rqosa (guest, #24136) [Link]

> There is just too much software and distros and not enough packagers. It does not scale to the point where all software is packaged for even one distro.

You might be interested in Arch Linux's Arch User Repository. This is a collection of user-contributed/maintained build scripts (PKGBUILDs) which can be used to build Arch packages. PKGBUILDs are fairly small and easy to develop: see one for MPlayer here. (Of course, a user building packages from the AUR should review the PKGBUILD and .install file if there is one for malicious code before building/installing.) The AUR web site is currently reporting that there are 19946 such packages in AUR (not counting ones that have been adopted into [community]).

topic: is Debian's packaging good or bad

Posted Feb 17, 2010 1:57 UTC (Wed) by rqosa (guest, #24136) [Link]

> For the majority of people installing software, especially when it comes to any sort of graphics or gaming or anything of that nature, are much better off using Windows to install and play around with open source software then Debian or Fedora or anybody else.

The main reason for that is probably just that there are more installations of Windows (on desktop and notebook computers at least) than there are of Linux distributions, so an app developer can get more users by building an installable package for Windows than by doing the same for Linux. (And also, building an installable package is basically the only way to get Windows users to use the software, because build tools aren't included with Windows.)

However, building such installable packages is a major pain for developers, because they have to package all library dependencies with their software (except for ones provided by Windows itself) and provide their own updater mechanism, so you have one updater for Firefox, another for VLC media player, another for OpenOffice, etc...

Is there any real reason why developers can't do the same for Linux (bundle all the dependencies with the application)? I don't think there's anything stopping them. The LSB is even supposed to provide a set of library ABIs which a program can depend on without having the libraries packaged with it. Yet, it seems like hardly any developers care to build their apps for the LSB, for some reason... (The Linux Foundation has also built a few example LSB packages of free apps here.)

topic: is Debian's packaging good or bad

Posted Feb 17, 2010 2:09 UTC (Wed) by rqosa (guest, #24136) [Link]

> bundle all the dependencies with the application

Or, use static linkage. (In the case of libraries which are normally dlopened, use Libtool's "Dlpreopening" capability.)

Applications not packaged by the distribution

Posted Feb 20, 2010 17:42 UTC (Sat) by anton (guest, #25547) [Link]

Yes, it can be quite a bit of work to install some applications (and all the supporting libraries etc.) from source. E.g., it can already take some time to figure out where to get the source from, and then whether to use the source repository head or the released package. One may not want to go to these lengths before knowing that the application is actually worth it.

One example was when I wanted to see if the new version of hugin was better than what was available through Debian at the time. I did not want to work for half an hour or an hour to try that out, so I downloaded and tried the Windows version, because that was much simpler to install. Only when I knew that it was worthwhile did I invest that effort.

Why was the Windows version simpler to install? Mainly because Windows is a less featureful system, where basically every application package has to do everything for itself, and it's therefore not considered improper for an application package to install all kinds of shared libraries that a more sophisticated system would have as a separate package. The other reason is that there is only one Windows version, whereas there would be one Linux package per distribution.

We could do the same for Linux, but then we would distribute tar packages with everything (including possibly outdated libraries) included, or maybe with statically linked binaries.

We don't want that for packages (at least package versions) supported by our distribution, because this approach has a number of disadvantages. But maybe it (or some other cross-distribution approach) would be appropriate for the applications that are not supported by all distributions.

That brings up the following idea in my mind: A cross-distribution package system (so packaging can be shared between distributions) that downloads sources and builds and installs them (to /usr/local or somesuch) mostly automatically. Maybe with an option for binary packages. Dependencies on libraries or programs provided with the distribution would be tricky, but it may be possible to satisfy them with Debian's "apt-file search" or similar functionality.

Building from source would avoid most ABI problems and would allow supporting many different hardware platforms, and allow the user to use the latest code without the packager having to upgrade the package for every source update (of course there will be some cases where the package will not work with the new sources, then the package will have to fall back to a known-good source, or the user has to install in the old-fashioned way).

Such a system would provide a rougher experience than the distribution's system (more frequent failure or breakage, not necessarily clean deinstallation), but it's better than having to do the installation manually (with the same disadvantages).