Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration [LWN.net]

On his blog, Vincent Untz reflects on the recently completed cross-distribution App Installer meeting, which by his and others' accounts was definitely a success. In the posting, he also spends some time talking about the need for more cross-distribution collaboration. "To be honest, since I started working on openSUSE, I've kept wondering why all distributions duplicate so much work. Sometimes, there is a good reason, like a radically different technical approach. But sometimes, it looks like we're going different ways just for the sake of doing something ourselves. We should fix this. Cross-distro collaboration is not the way we usually do things, and I believe we're wrong most of the time. Cross-distro collaboration is a cultural shift for us. But it's very well needed."

to post comments

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 2:57 UTC (Wed) by skvidal (guest, #3094) [Link] (23 responses)

package/app install/updating is critical to how a distro does everything it does. So to suggest more collaboration between distros is to, implicitly suggest collapsing distros together.

What's the major distinguishing factor between debian and fedora?

dpkg vs rpm.

Why would debian want to switch to rpm? What does it do for them to have to retrofit a million things into new processes and new behaviors?

Why would fedora want to switch to dpkg? What does it do for them to retrofit a million things into new processes and new behaviors?

So, what is the incentive for a distro that wants to survive and be independent to collaborate on this kind of technology?

There has to be a big win to the technology to promote collaboration and thus far there's been nothing.

I suspect that the pain of migration will continue to outweigh the benefit of the technology and what is more likely to happen is that more and more of the major distros will die out due to external economic realities until the options are winnowed down. Then cross-distro collaboration is less important.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 3:27 UTC (Wed) by yarikoptic (guest, #36795) [Link] (7 responses)

> So, what is the incentive for a distro that wants to survive and be independent to collaborate on this kind of technology?

Because there is LOTS of effort duplication going on and joining the forces would benefit all communities and companies staying behind (echoing your 'Debian/Fedora difference question'). Neither distribution is suicidal enough to artificially create "differences" just for the sake of staying "different"; but it historically happened that different approaches/instrumentations/ideologies came into existence and now split the "distributions" market and userbase.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 3:38 UTC (Wed) by dlang (guest, #313) [Link] (1 responses)

there is work that can be re-used (figuring out what apps and library versions a particular application needs for example), and there are parts that cannot (the final packaging into .deb or .rpm)

keep in mind that there are applications that do a pretty good job of converting between .deb and .rpm today, that could be enhanced, or a new metadata format could be created that contains a superset of the data that the different package formats need (making the job of creating packages for multiple distros easier)

also, feeding patches between distros and to the upstream will also help reduce duplicated effort.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 22:07 UTC (Wed) by vonbrand (guest, #4458) [Link]

there is work that can be re-used (figuring out what apps and library versions a particular application needs for example)

Most of that is done automatically today. E.g. in Fedora the packaging guidelines specify that only in rare cases the packager should give dependencies explicitly.

keep in mind that there are applications that do a pretty good job of converting between .deb and .rpm today, that could be enhanced, or a new metadata format could be created that contains a superset of the data that the different package formats need (making the job of creating packages for multiple distros easier)

It isn't just a "format" issue, the differences are in packaging policies (i.e., what packages are named, how much to split up packages for libraries, handling of documentation). And futhermore there are differences in configuration handling (e.g., how/if default configurations are handled, where configuration is kept, how it is handled).

also, feeding patches between distros and to the upstream will also help reduce duplicated effort.

That is a distribution policy issue. I believe distributions have gotten ever better at working (with) upstream, keeping local patches only where strictly necessary (or as an interim fix while upstream fixes their code).

Disclaimer: I'm a Fedora user and ambassador, so the above might certainly be biased by the distribution I do know best.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 4:10 UTC (Wed) by nevyn (guest, #33129) [Link] (4 responses)

> Because there is LOTS of effort duplication going on and joining the forces would benefit all
> communities and companies staying behind

Sure, and there's lots of effort duplication between Qt+KDE and gtk+GNOME ... and with GNOME3 and Ubuntu that looks like it might split again.

As Seth said, the problem is that after the initial split and time investment it's just _really_ hard to merge back. And to create "a standard" that multiple people can implement to easily, is at least as hard as everyone just doing their own thing with very few benefits (what users will care that backend format X is shared with distro. FOO which they'll never use).

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 4:29 UTC (Wed) by dlang (guest, #313) [Link] (2 responses)

I'll point out that KDE and Gnome are cooperating, that's what freedesktop.org is all about, they are working to define the underlying pieces in a way that they can both live with, even if the implementation and visible result is significantly different between the two.

Ubuntu is straying from this a bit, but unless they get buy-in from lots of application developers to write to a different, Ubuntu specific API, it won't stay separate long term (it may be that Ubuntu moves back to mechanisms that currently exist, or it may be that some of the things that Ubuntu is trying become standard and the other desktops start supporting them)

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 15:46 UTC (Wed) by nix (subscriber, #2304) [Link]

freedesktop.org often looks a lot more like 'GNOME people define stuff, KDE people use it'. GNOME people get their stuff in without effort: when KDE people try, all too often there is endless argument and nothing happens.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 28, 2011 4:08 UTC (Fri) by markshuttle (guest, #22379) [Link]

Much of the Unity work in Ubuntu is based on FreeDesktop.org collaboration with, amongst others KDE, and it's Gnome which has decided to purse a different course.

The Ubuntu indicators framework, for example, was developed in Ubuntu after discussions with Gnome developers, and was also developed in collaboration with KDE developers. When it was ready, Gnome Shell said they had decided to do something different altogether. It's a little perverse to be accused of being uncollaborative when you:

o discuss work in advance with the Gnome team and get a go-ahead to drive forward
o collaborate with other communities like KDE in designing the framework
o deliver what you promised
o then get told that Gnome's designers have changed their mind and will pursue a course that reflects neither the agreement nor a new consensus across platforms ;-), and
o get rejected when you propose the framework for inclusion as a Gnome external dependency!

The dynamics of collaboration in open source are fluid and complex. There's a lot that happens which is constructive, and there are also things that make everyone want to despair. But it's simply inaccurate to paint one group as a primary bad guy; much just depends on who was where at the time.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 9:39 UTC (Wed) by oldtomas (guest, #72579) [Link]

Yes. As dlang said, there are many creative ways projects can cooperate, and things become really interesting in the realm of Free Software. The important part is to stay in touch and try to talk to others while implementing new stuff (watch, for example, the Emacs vs. XEmacs cooperation these days. When introducing changes, thought goes into how those changes can benefit both).

Things which come to mind cross-distro:

* Abstract package structure (which attributes, what's a dependency, etc.)
* Translations (why has Puppy Linux so few translations compared to Debian. They are free to use, after all. What can be done about that)

No need to be directly able to install an RPM in Debian (although even this is somewhat possible these days!)

dpkg v rpm

Posted Jan 26, 2011 4:27 UTC (Wed) by corbet (editor, #1) [Link] (4 responses)

Interesting...when I think about the differences between Debian and Fedora, the package manager is mostly an incidental detail. Sort of like saying the main difference between Australia and the US is that they drive on different sides of the street. Things like release process, how the community works, project governance, etc. strike me as being much more important.

dpkg v rpm

Posted Jan 26, 2011 7:39 UTC (Wed) by skvidal (guest, #3094) [Link] (3 responses)

How does project governance matters to how the OS works?

B/c that's what we're talking about.

Not how the community hangs together (or not) as the case may be.

we're talking about the processes to:
1. compose the distro (the os)
2. organize the distro (the os)
3. deploy/install the distro (the os)

it's all coupled tightly to the packaging tools.

The os itself is the same ON PURPOSE.

apache on debian and apache on fedora are, by and large, the same.

The set of policies/rules are just the sausage-making process.

Take a distro to its component parts of the OS:
1. kernel -pretty much 'upstream' + 'whatever'
2. basic 'get you booted' utilities.
3. apps/daemons/etc
4. packaging

parts 1, 2, and 3 need to be very similar for us to call it 'linux' don't they?

Packaging/installation/maintenance is the separation point.
that's the part that makes us put another name in front of linux:

Debian
Fedora
OpenSuse

So, I don't generally group rpm and deb as the split. It's more like 'families' of distros.

debian is one family of distros including: debian and ubuntu and all the derivatives thereof.

red hat is another including: rhel, fedora, centos, etc.

SuSE is another including: SLED/SLES and OpenSUSE

Mandriva is another.

The tools that tie each of these together are their installation/update/pkging tools.

Those are the tools the admin/user end up having to be most familiar with.

That's what I mean when I say the packaging is where things vary.

dpkg v rpm

Posted Jan 26, 2011 14:16 UTC (Wed) by sorpigal (subscriber, #36106) [Link] (1 responses)

For me the most important differences between a Debian system and a Fedora system aren't package format (no one cares) or package manager (even though apt is better!) or package related tools. The important differences are the results of policies that are different.

How are network interfaces configured? What about firewalls? Where do defaults get set? What tools control runlevel services? What is the layout of /var/? Where is documentation stored? What can be found in /usr/local/? If you have a bug, what do you do? Everything here is different and has nothing to do with packaging as such. In some cases such decisions can be made by the user by installing a different package from the default, but then which was the default was a distro policy decision.

I'm all for working together on things not related to packaging, but I use Debian because I tend to like the design decisions Debian makes. I don't want to see other distribution's decisions creeping in to mine (and I'm sure this is mutual).

dpkg v rpm

Posted Jan 26, 2011 19:19 UTC (Wed) by martinfick (subscriber, #4455) [Link]

Agreed. +2

dpkg v rpm

Posted Jan 26, 2011 17:58 UTC (Wed) by iabervon (subscriber, #722) [Link]

Take a distro to its component parts of the OS:
1. kernel -pretty much 'upstream' + 'whatever'
2. basic 'get you booted' utilities.
3. apps/daemons/etc
4. packaging

parts 1, 2, and 3 need to be very similar for us to call it 'linux' don't they?

Parts 1 and 2 are generally very similar (although the initial filesystem stuff was, at least until recently, totally custom for each distro). But part 3 varies widely between distros, at least with respect to the willingness to experiment with alternatives and be early adopters. Distros varied in when they started using Pulseaudio; what replacement(s) for sysv init they tried; how they set up networking; the support code for init scripts; when they adopted what of hal/dbus/udev/ConsoleKit/PolicyKit; the default mail server; the default mailbox file format; and so on, down to whether they ship a version of Alpine that outputs utf-8 or are too concerned about the legal status of those patches. And, of course, these adoption decisions are made using the distro's project governance process.

Furthermore, these differences trickle into the packages; when one distro is using upstart and another is using sysv init, their apache packages have to include different scripts to manage getting apache to run, and when a third distro is also using upstart, there isn't a way to avoid duplicating the uninteresting work of maintaining the script for the apache package.

That is, the differentiation between distros is a bunch of choices the distro makes that matter in small but critical ways to the packages they ultimately provide. This forces them to maintain their own packages independently, and the individualized packages are a cost, not a benefit, of the differentiation.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 4:34 UTC (Wed) by elanthis (guest, #6227) [Link] (3 responses)

"dpkg vs rpm."

That is the single least important detail there is.

I suppose you think Debian and Ubuntu are identical in every way, or that Fedora and Mandrake and openSUSE are identical in every way, because their greatest difference is something less important than whether you type 'dpkg' or 'rpm' to query packages.

There's no noteworthy difference between dpkg and rpm. They are functionally equivalent in just about every important way. The only times the user is even made aware of the difference is when downloading packages, due to the artificial barriers to compatibility imposed by the packaging systems, forcing them to have to pick the right download link using archaic knowledge about system implementation detals. But then, you run into that same problem trying to download a random RPM because the distros add all kinds of other arbitrary and artificial compatibility barriers that serve absolutely no purpose at all but which make it so a single RPM may not even install on different releases of the exact same distribution.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 11:50 UTC (Wed) by dgm (subscriber, #49227) [Link]

Agreed. Even package availability doesn't mater much. How many Ubuntu derivatives are there? All share the exact same repositories (and thus package format), yet each has it's own personal twist.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 14:09 UTC (Wed) by Seegras (guest, #20463) [Link] (1 responses)

> There's no noteworthy difference between dpkg and rpm. They are
> functionally equivalent in just about every important way.

Nope. dpkg had dependencies on packages, whereas rpm has dependencies on libraries/programs. Thus rpm needs a database to look up which package contains a dependency, whereas dpkg does not.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 14:50 UTC (Wed) by Darkmere (subscriber, #53695) [Link]

Actually, RPM has _both_ kinds of dependencies. Both listing filenames, and package/program-names.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 10:31 UTC (Wed) by renox (guest, #23785) [Link] (2 responses)

>> package/app install/updating is critical to how a distro does everything it does. So to suggest more collaboration between distros is to, implicitly suggest collapsing distros together. <<

A baseless assertion: there is a large number of combinations of application and customisations which can be chosen by the distributions!
And these choices are much more important for the users than the package mechanism used by the distribution.

More simply: I don't care at all what type of package the distribution use, I do care that chromium is available and well integrated..

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 12:42 UTC (Wed) by drag (guest, #31333) [Link] (1 responses)

Yes. All the same distros releases around the same time period use the same kernel versions, gcc versions, X versions... etc etc.

They have a hell of a lot in common. Much more then have in differences.

A lot of the package and software incompatibilities are just caused by relatively insignificant implementation details. Stuff like install location or how the packages are divided up into parts or the versioning scheme used.

Companies and projects that support installing software across multiple distros don't bother compiling them statically or anything weird like that. They will include some libs, but often only because they need special features or versions not typically included by the distros.

For the most part I know I can 90% of the software built for Fedora and have no issues running it on Ubuntu. The biggest barrier is the package management itself.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 19:26 UTC (Wed) by martinfick (subscriber, #4455) [Link]

> A lot of the package and software incompatibilities are just caused by relatively insignificant implementation details. Stuff like install location or how the packages are divided up into parts or the versioning scheme used.

Insignificant? But, those are exactly the most important things that make a distribution a distribution. Without policies which force unrelated software to work together on a system you are left with the nightmare that you see in windows. The whole point of a distribution is for the package maintainers to think about these things for you. They are the added value of a distribution over simply downloading and installing upstream packages.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 15:35 UTC (Wed) by xxiao (guest, #9631) [Link]

maybe we should unify all distros, make it a single OS, something like windows, users just switch themes to differentiate their system?

in reality, however, this is pointless and nothing new.

should Suse/Mandriva have used deb instead of rpm, maybe there is no ubuntu today...

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 18:32 UTC (Wed) by vonbrand (guest, #4458) [Link]

There are deeper differences than just RPM vs dpkg. For example, SUSE packages (while also RPM) couldn't be installed on Fedora because the division of e.g. library packages were wildly different, and the package names didn't match up (for resolving dependencies), and often the location and names of the configuration files was different. Even among Red Hat derived distributions (like Mandrake and Fedora) it wasn't easy, as they used different library versions and compiler flags. The safest bet was to get the source RPM for a foreign distribution, dissect that one to create a native binary RPM (perhaps by hacking the patches or updates from the foreign one into to native source package). The big advantage of dpkg is that all the dpkg based distributions track Debian rather closely.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 28, 2011 16:56 UTC (Fri) by eean (subscriber, #50420) [Link]

dpkg, rpm are of course different, but they aren't that different. They solve the same problem with much of the same methods. There's no reason for a dpkg-only or a rpm-only installation GUI to exist.

so... what are you even talking about?

IMO that 99+% of the work done upstream is the cross-distro collaboration

Posted Jan 26, 2011 5:33 UTC (Wed) by filteredperception (guest, #5692) [Link] (1 responses)

In my mind, when I look at the totality of work that went into producing the open source distro in front of me that I'm using, I see 99+% of that work as the code that is upstream, and already shared, and thus collaborated between all the distros. I think that this complaint about lack of collaboration is failing to acknowledge the overwhelming majority of collaboration that is already in place in the form of shared upstreams.

And when I look at the bigger differences, I don't see a wasted duplication of effort, but an organic competition of options/traits in what is effectively a gene-pool of overall system defining ideas.

Consider any other craft, like say, people who build guitars. Is there a lot of duplicated effort, and wouldn't things be more efficient if there were just one assembly-line making a bunch of guitars that all satisfy at least 95% of the needs of all the existing users? Sure, maybe that would be more efficient, but it would also be a really boring solution, versus one which supports lots of independent craftspeople all getting the fun of building their own slightly different guitars.

IMO that 99+% of the work done upstream is the cross-distro collaboration

Posted Jan 28, 2011 18:38 UTC (Fri) by eean (subscriber, #50420) [Link]

Distros are their own "upstream" traditionally for system configuration utilities (think YaST), application install GUIs (what this sprint was about) and of course the low-level package tools (zypper, dpkg and what comments in this thread are obsessed with).

With the first two groups there is a lot of needless duplicate work. There are certainly cross-distro projects in those areas (NetworkManager comes to mind; its mostly "distro devs" not upstream devs that work on it), but more work could be done.

Its fine to have a software ecosystem of competing projects, but there is a finite supply of open source developer hours. They should cooperate when they can.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 5:54 UTC (Wed) by dlang (guest, #313) [Link] (2 responses)

looking at the agenda for the meeting, the very first thing on the agenda should not have been there (agree on a common UI for installing software)

trying to define a common UI across all distros is very much the wrong thing to do. A UI that is good for a high-powered machine with a 1080P display is going to be horrible on a phone or tablet with a 800x600 (or less) display and vice versa. and when you consider server distros where you may only have a serial console, the idea of standardizing the UI is even sillier.

If the people who attended managed to reach agreement on this issue, it's a clear signal that they don't have a clue and should be ignored.

the fact that this was one of 4 items on the agenda indicates that whoever created the agenda should not be trusted in this area.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 6:21 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link]

Your objections don't take into consideration the context. The UI that is being discussed is only for the desktop. Not for tablets of servers or whatever else.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 2:18 UTC (Thu) by vonbrand (guest, #4458) [Link]

looking at the agenda for the meeting, the very first thing on the agenda should not have been there (agree on a common UI for installing software)

Isn't that called PackageKit?

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 6:36 UTC (Wed) by xav (guest, #18536) [Link] (23 responses)

A common "app installer" looks to me just like a way of bypassing all the efforts distro put into distros packaging. Nowadays you have a coherent set of packages, with matching versions, security audit, and maintenance. With an "app installer" you're straight into Windows world with random spyware-laden apps installable with a simple click.
I don't think it's a progress for Free Software in general.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 6:59 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link] (13 responses)

That makes no sense. App installer is merely a abstraction layer over the native package manager.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 9:08 UTC (Wed) by xav (guest, #18536) [Link] (12 responses)

This has nothing to do with the technical side of installation. Let me try to explain my point of view to you:
- when running e.g. Debian, Ubuntu or Fedora, the end-user has an expectation of stability and security
- that's because the pool of package managers is known, they all talk together and respect a set of common rules, and are reactive to events (e.g. security alerts)
- and that's because the end-user is sort-of restricted in what it can install on its machine, that is packages from the distribution; only so-called power users will install alternative packages
- now you want anyone being able to set up a blinking webpage telling users "install that shiny new GonzoMediaPlayer to be able to watch football matches for free"
- there you are, your rock stable linux distribution is now down to a Windows machine, with anyone being able to distribute malware for any distribution, and unknowledgeable end-users' systems plagued with it

I see the fact that executables must go through a distro-specific packaging process first as a virtue, not a defect. Apparently not everyone agree.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 9:52 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link] (9 responses)

Clicking on a web page doesn't fundamentally change the process. It can just be a call to a repository. Besides, it is illusion to think that end users, especially desktop users just stick with the apps in the repository or that a single repository can provide everything a user wants. There are tons of apps or apps in the version that users want that are outside any mainstream Linux distribution repository. They go click and add random repositories all the time and that is really no different than clicking a random web page to install a app. If desktop Linux becomes mainstream, we are going to have to tackle the malware problem differently than trying to enforce a one repository rule.

Let's look at one aspect of the problem with the current model: If Firefox 4 releases tomorrow and I am using Windows or Mac OS X, I can install it immediately and in Linux, one usually has to upgrade their entire distribution. In Linux, there is no fundamental differentiation between system packages and even leaf applications. This is a problem that one has to solve and the current methods are awfully weak. The system admin point of view is great for servers but desktops need a different solution.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 10:08 UTC (Wed) by niner (guest, #26151) [Link] (1 responses)

What you describe already exists. It's openSUSE's One-click-install. You click a link and the handler subscribes you to the repository and installs the requested package along with it's dependencies. You can then select if you want to stay subscribed to that repository to for example receive future updates.

Makes staying up to date with for example Firefox trivial.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 10:36 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link]

I am aware of openSUSE's solution as well as others like Ubuntu's PPA etc but it has to be available for more than just one distribution. That is sort of the point of getting the different distributions together and hash out the problems and solutions.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 11:41 UTC (Wed) by nicooo (guest, #69134) [Link] (1 responses)

>Let's look at one aspect of the problem with the current model: If Firefox 4 releases tomorrow and I am using Windows or Mac OS X, I can install it immediately and in Linux, one usually has to upgrade their entire distribution.

Which model are you referring to? I'm using gentoo and this works fine. The nice thing about distributions is that there are different models.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 12:15 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link]

It thought it was obvious that I was referring to the mainstream distributions that are part of the discussion in the article. The problem with the different models is that it very much a pick your poison kind of thing. Gentoo isn't something one would recommend even if the users like having the latest applications.

one-click third-party software installs

Posted Jan 26, 2011 21:10 UTC (Wed) by jthill (subscriber, #56558) [Link]

I think it's plain this notion produces exactly the same sequence as what produces the Windows nightmare: anyone can construct a web page or email and put anything at all at the fetch URL. No.

Clicking on a web-page does fundamentally change the process. It's a little weird to newbies. It takes manually fiddling with system files, unless somebody's already made an update-sources.list protocol. imho that's just barely enough to stave off the Windows nightmare now.

Because installing third party software with apt means running root scripts directly from whatever's behind the flashy link, does it not?

I think a third-party install scheme that protects the system and the user has to be cooked up before considering this.

Say, for third-party-signed packages create a new user/group combo, that can be tied to an "origin" key that must also have signed the package-signing key. For unsigned packags just use "nobody" or "third-party" or something, they don't care enough they can go in the swamp.

Something to that effect, for sure:: you want to also protect these packages from each other.

Run the package scripts on the package's userid, with a special service to install links into the system hierarchy. Apps will also want some writable .config or .local or whatever it is now subdirectory, also prompt users to grant access. Put the app's per-user config one level down, .config/example/example-config, so metadata is inaccessible to the app. Log a copy of the outside-/opt actions to syslog, and show them to the user in like a three line text box with a that's all indicator. Offer "here's what's installed on your system. I've logged it to syslog. Want a copy?" <Save install log> <Thanks, I'm good>.

The access restrictions will stop most if not all of the Windows uninstall nightmare. Say add a product key in addition to the package key, then tag the directories with the responsible key. .config/opt/foo/.PRODUCT_ID_fingerprint doesn't match any installed package's fingerprint? It's stale. Make the tag file owned by root, the product's group, 744 so the user and app can verify, another service to update contents from newly-installed packages from the same origin. That's easily subvertible by the user

That, or convert everyone to kerberos...

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 0:17 UTC (Thu) by garloff (subscriber, #319) [Link] (3 responses)

> If Firefox 4 releases tomorrow and I am using Windows or Mac OS X, I can
> install it immediately and in Linux, one usually has to upgrade their
> entire distribution.

You happen to not have spotted openSUSE Build Service yet.
It's dead simple to build packages for various distributions that you can simply drop in to older distros.

The main limiting factor here is that once you build on newer shared
infrastructure it becomes more tedious ... but the FFox4 example is one
where this works rather easily.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 0:23 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

I am aware of that as I have already indicated before but If I am a end user, I don't want to be building packages in any build service. That's just ridiculous. If it is not in my distribution repository then I just want to install Firefox 4 from a obvious place. mozilla.org would the most obvious location.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 0:27 UTC (Thu) by garloff (subscriber, #319) [Link] (1 responses)

Well, I don't suggest end users should build most of their software themselves. But someone can do it. Most obvious choice would be the
upstream community itself.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 1:57 UTC (Thu) by vonbrand (guest, #4458) [Link]

Upstream has enough work cut out for them just getting the base to work (and fix bugs), asking them to also track whatever a few hundred (eo even just a dozen "mainstream") distributions are up to is just over the top.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 14:11 UTC (Wed) by sorpigal (subscriber, #36106) [Link] (1 responses)

Distro package repositories are nice but cannot scale to "everything." A solution that doesn't devolve into the Windows mess is possible; I think this is a useful related discussion.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 2:28 UTC (Thu) by vonbrand (guest, #4458) [Link]

Part of the benefit of distribution repositories is exactly that they don't package everything: It is a way of filtering out crap.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 7:46 UTC (Wed) by skvidal (guest, #3094) [Link] (8 responses)

I think you're exactly right and from the grapevine I know this was raised.

Deep down a number of folks want an 'app store' just like osx has, of course.

And they want to be able to ship 'bundles' so they don't have to worry about things like dependencies at all.

What everyone fails to remember is how much of a godawful nightmare bundles are to maintain.

Chase down a zlib security bug through 1000 individually pkged versions of zlib in the bundles on your systems. It's a disaster area.

Everyone remember chasing down the timezone change 4 yrs ago in every copy of java that was littered all over your system? Think about that but for EVERY app and having to figure out if this is zlib 1.2.3 or actually 1.1.9.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 10:57 UTC (Wed) by mjthayer (guest, #39183) [Link] (7 responses)

> What everyone fails to remember is how much of a godawful nightmare bundles are to maintain.

> Chase down a zlib security bug through 1000 individually pkged versions of zlib in the bundles on your systems. It's a disaster area.

What I always wonder re this objection is why a bundle has to be done the same at source and binary levels. In the hope of clarifying that, imagine a Gentoo-like source distribution and a system of building bundles out of that which pulls all the necessary source packages, compiles them and bundles them. Then propagating the zlib security fix is as simple as (something like) "emerge bundleworld". That said, you can't put all dependencies in a bundle (well you can, but then it becomes a virtual machine appliance instead), and if you accept that anyway it might make sense to make things which are updated a lot for security (don't think zlib is in this category) into external dependencies to save re-downloading your whole system every week.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 12:05 UTC (Wed) by dgm (subscriber, #49227) [Link] (6 responses)

If you could do that it would mean you had proper dependencies before bundling. Why throw away all that information? Wouldn't it be better to just ship the packages unbundled and update as usual?

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 13:02 UTC (Wed) by tzafrir (subscriber, #11501) [Link] (1 responses)

So Firefox should not bundle Xulrunner, and you should wait for Thunderbird to be ported to the new Xulrunner, or remove it from your system.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 2:26 UTC (Thu) by vonbrand (guest, #4458) [Link]

So you ask for collaboration between distributions to be gotten by fobidding upstream packages to collaborate and share pieces... way to go.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 13:02 UTC (Wed) by kleptog (subscriber, #1183) [Link] (1 responses)

I think the real problem is that the app developers know the dependencies but have no way of ensuring that they're installed. Even if you know the package name in a particular distribution, there's no guarantee the version you want even exists there (see endless versions of libssl for example).

Binary compatibility is really hard and most people don't bother. About the only library I know that pays any attention to it glibc, which goes to great lengths to ensure you get the symbol you linked against rather than whatever version is newest. Most libraries just increase the version number and ship it.

Many of the problems are solved. For example, we all more or less agree on the FHS so files end up in more or less the same place. LSB was in principle a great idea, but obsolete before it was done. What you need is something moves much faster.

Lets suppose that a few of the faster distributions (e.g. Mandriva, Ubuntu & Fedora) got together once a year and drew up a list of the 50 most common libraries and agreed on versions they would ship. Each distribution would be free to provide other versions, as long as that particular version would also ship. This might help (or not :) ).

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 14:27 UTC (Wed) by sorpigal (subscriber, #36106) [Link]

This is the old "we need a common platform" argument and whether it's true or not doesn't seem to be important because we don't seem any closer, today, to actually doing it than we were ten years ago. I don't know that you can get cats to march in a line that straight and I think other solutions should be explored instead.

Creating platforms is a problem that has been up to now in the domain of the distribution. Most non-enterprise distributions don't actually worry about it at all, much less do it (except by chance). To have a third party dictate (or suggest) a platform to distributions is hard because (a) the distributions don't know what a platform is and (b) they won't like dancing to someone else's tune, especially if the advertised payoff is "More work for you but less work for people who are not you."

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 14:34 UTC (Wed) by mjthayer (guest, #39183) [Link] (1 responses)

> If you could do that it would mean you had proper dependencies before bundling. Why throw away all that information? Wouldn't it be better to just ship the packages unbundled and update as usual?

This is just my answer, not an answer from a wise person who has thought much about the matter. Mainly, I would say, because it reduces complexity. A FLOSS-style binary distribution requires masses of testing to ensure that every component works well with every other. Using bundles with reduced dependencies reduces the testing burden on the maintainer, even before you worry about cross-distribution installation.

Then there is the traditional argument that bundling dependencies frees you from worrying what is or is not available on the user's distribution. You just bundle it yourself, test your package and are happy.

Plus my personal feeling is that the package database on a FLOSS system is always a potential place for things to go wrong. Thankfully I think this has only hit me once (or maybe twice), but if it does get corrupt you are in for a massive fixing (or re-installing) session that would be much less painful on a system mainly using bundles. And from a cleanliness point of view, it is much easier to have an overview of what a system mainly based on bundles has on it. I must admit that that last point is what I find most attractive about bundles - they give you much more of a feeling of being in control of your system while (by) structuring (if not reducing) its overall complexity.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 2:17 UTC (Thu) by vonbrand (guest, #4458) [Link]

Bundles are either massive duplication (and innumerable opportunities for security (and other) bugs that get never fixed) or require something like the current dependency handling system, but on weapons-grade steroids (you'd have to track not only one set of dependencies, but potentially dependencies for each single package in the system separately). I just don't see anything but downsides.

Yes, the package database is a single point of failure for a Linux distribution. Yes, in the 15 or so years of Red Hat (and derivatives) use I did have my half dozen problems with broken RPM databases, but AFAIR just once there was no reasonable way to fix it without a full reinstall (and that was due to a disk problem, that thrashed the system regardless). Sure, fixing the database wasn't trivial; but the technology used has gotten way better over time, the last thrashed RPM database was quite some time back.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 12:33 UTC (Wed) by djzort (guest, #57189) [Link] (5 responses)

as soon as redhat (and hence fedora) gives up on rpm and its half baked apt clone - yum, this sort of discussion will be pointless.

rpm was a great idea, but debian took things to the next level by wrapping dpkg up with apt. why redhat clings to rpm+yum is beyond reasonable comprehension. most likely its rooted in redhats irrational fetish for python.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 12:55 UTC (Wed) by Darkmere (subscriber, #53695) [Link] (3 responses)

Do enlighten me, does dpkg still allow(!) the package to force interactivity during installation/update? As in, does it still have the ability to completely destroy automatic updates?

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 13:18 UTC (Wed) by nye (guest, #51576) [Link] (2 responses)

Not sure if it *can* force it, but I've been updating automatically (using DEBIAN_FRONTEND=noninteractive) for over a year now and I've never come across a situation where it *does* force it.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 14:29 UTC (Wed) by sorpigal (subscriber, #36106) [Link] (1 responses)

If by "force interactivity" you mean "prompt you by default if you don't know how to tell it not to" then the answer is "Yes." That's probably what he meant.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 14:43 UTC (Wed) by DOT (subscriber, #58786) [Link]

What concept do you think the word "default" relates to: format or configuration?

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 27, 2011 2:23 UTC (Thu) by vonbrand (guest, #4458) [Link]

As soon as Debian finally wises up and embraces the (standard!) RPM packaging format and the relevant tools...

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 12:47 UTC (Wed) by jjs (guest, #10315) [Link] (1 responses)

@rahulsundaram -
Not certain what you call "mainstream" but for Debian, if I want to install iceweasel (debian's version of Firefox) from experimental, I type in "apt-get -f install iceweasel/experimental" and the system donwloads iceweasel and all dependancies. No other upgrades. My experience with Fedora and Red Hat is the same using yum.

Only Windows requires complete upgrades and restarts.

Untz: Results of the App Installer meeting, and some thoughts on cross-distro collaboration

Posted Jan 26, 2011 12:55 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link]

xulrunner is packaged separately in both Debian and Fedora and everytime the ABI changes, several other packages that depend on it has to be rebuild and updated as well and hence if you pick a newer version of Firefox from a experimental repository, you will get other updates as well. The typical solution in other platforms is bundling and while that can be done in Linux, distributions explicitly advocate against that and hence you will end up cascading updates in many cases. Besides, telling users to install from a "experimental" repository doesn't help either.

Isn't this just an extension of the Linux Standard Base?

Posted Jan 26, 2011 19:04 UTC (Wed) by southey (guest, #9466) [Link]

Without something like the Linux Standard Base you would end up with some huge files to account for the differences between distros as well as the within distro differences (such as different releases or patch state). Or something that generates the appropriate files (perhaps like alien tries to do).

what I would love

Posted Jan 26, 2011 23:28 UTC (Wed) by albertoafn (guest, #64225) [Link] (4 responses)

When I install an application I would love to have a standarized description with some vital information (at least what I consider vital)
I.e.:
- languages available
- dates (year of 1st release, date of 3 last updates)
- version
- history (when applicable)
- dependencis (Installed and new)
- works fine with gonme/kde/xcf/others
- cli? gui?
- related guis availables
- tasks that can be done with the program
- supported formats (when applicable)
- useful paths for the application (where the * is the config file? where does it save data by default, etc...)
- is it dead? (ie. is there any known ppl/team working on it? related pages? last time the repository version was update?... etc...)
- description
- a liltle video of the application working (if applicable, ie. games) and date/version the snapshot/video was done
- ... some extra stuff that is only applicable to some application but can be useful to know (ie: related projects, or special trick for that application)
[...]

All this can be of course be turn on or off (i.e simple or advance. Maybe some + to see more information for the app)

Me, and I think most of us, would have their brains blown up by the awesome of a package system like this. And this little features are not hard at all to achive :)

what I would love

Posted Jan 27, 2011 2:49 UTC (Thu) by vonbrand (guest, #4458) [Link]

It is just a huge amount of work for the developers of the packages (who probably do their work only because they enjoy a rather different set of challenges) just for satisfying the idle curiosity of a few random browsers. Won't happen.

That without even going into how do you get the developers to agree on a standard way of describing their packages. The closest thing today are the webpages of the packages. Now just look at the web pages for, say Gnome and TeX, or GCC and LibreOffice, and say what the similarities are, and how you can force all the different data into a common format. TeX or GCC won't have any use for a video of its "user interface" (which one?), while for Gnome or Libreoffice it could be important. The "input format" for TeX and GCC is given, either by the package itself or external definition. Today either a package does work fine with all the major graphical environments, or none; it would be probably much more informative to tell if it works with some of the niche ones (and I don't want to have to check something out on an environment that I barely know). For (La)TeX there are literally dozens of GUIs available, on a huge variety of systems. Same for GCC. For TeX there are lots of websites and newsgroups, even publications, around tips and tricks. Which one to pick to suggest in such a case?

what I would love

Posted Jan 27, 2011 13:03 UTC (Thu) by pboddie (guest, #50784) [Link] (2 responses)

When I install an application I would love to have a standarized description with some vital information (at least what I consider vital)

Aside from completely independent sites such as Freshmeat which have been attempting to catalogue software for many years already, there has been a certain amount of movement towards augmenting the basic metadata within various services. I think that the Debian package archive's Web site now has screenshot support, for example.

That said, some of what you've mentioned is useful information, even that which isn't already recorded in some fashion.

I.e.:
- languages available
- dates (year of 1st release, date of 3 last updates)

The latter is just a matter of a repository search, more or less.

- version
- history (when applicable)
- dependencis (Installed and new)

All standard package metadata.

- works fine with gonme/kde/xcf/others
- cli? gui?
- related guis availables

Supported to an extent with metadata and "suggested packages".

- tasks that can be done with the program

To an extent, the Debian dependencies support this, although it's largely to do with whether a program provides the same facilities as another program.

- supported formats (when applicable)

Interesting!

- useful paths for the application (where the * is the config file? where does it save data by default, etc...)

This is a policy matter: the configuration files, for example, should all reside in a standard place such as /etc/sysconfig (or your distribution's alternative). It would be nice to be reminded of this via the package manager though...

dpkg --show-config-files <package>

- is it dead? (ie. is there any known ppl/team working on it? related pages? last time the repository version was update?... etc...)

More a searching or statistics gathering activity, like the frequency of releases mentioned above.

- description

Very standard metadata, this!

- a liltle video of the application working (if applicable, ie. games) and date/version the snapshot/video was done
- ... some extra stuff that is only applicable to some application but can be useful to know (ie: related projects, or special trick for that application) [...]

This is the sort of thing that Web site front-ends attempt to provide, even today.

All this can be of course be turn on or off (i.e simple or advance. Maybe some + to see more information for the app) Me, and I think most of us, would have their brains blown up by the awesome of a package system like this. And this little features are not hard at all to achive :)

Many of them are already there, so I'd rather have my mind blown by something else, really. ;-)

what I would love

Posted Jan 27, 2011 17:46 UTC (Thu) by albertoafn (guest, #64225) [Link] (1 responses)

     Aside from completely independent sites such as Freshmeat which have been attempting to catalogue software for many years already, there has been a certain amount of movement towards augmenting the basic metadata within various services. I think that the Debian package archive's Web site now has screenshot support, for example.

For me its not the same freshmeat and a repository. A repository has some sort of control over what i am installing. Not just random code (or at least that's what i'd like to think)

Debian has the screenshot support. It doesn't tell you the version of the screenshot tho :(

[...] Anyway, yeah, some of this information is already available. Usually the faster way to find out about it, it's find out the project homepage somewhere or google it and look for it there. That or a few commands to find out about this metadata, but with a generic form like this to fill in by maintainers would be much centralized and much of the data only has to be entered once...

This is for example where the large base of "noobs" users of ubuntu that everybody like to bash can come in handy to help gather all that data. Its not complicated data (NOTE: no offense intended. noob = not as savvy as other distributions. I know lot of people that use ubuntu that would love to collaborate but does not know much, even to triage bugs)

 This is a policy matter: the configuration files, for example, should all reside in a standard place such as /etc/sysconfig (or your distribution's alternative). It would be nice to be reminded of this via the package manager though...
dpkg --show-config-files package

I meant all kind of useful path or env variables, not just conf files. And yeah its a matter of the policy, but thats what they are trying to sort it out in this article, isnt it? :)

btw when i said history i meant changelog :)

Many of them are already there, so I'd rather have my mind blown by something else, really. ;-)

I agree that many are there, but not in a uniform accessible way. Id love to see all this things in a gui with the ability to sort the packages by these fields and compare between each other

And many of this would make more people to be able to use repositories. Yeah, nowadays I can tell my gf to look for photoshop or gimp and she is able to install it. But how can she pick her own packages without installing it and try a few out? with more metadata presented in a uniform way, she would start loving repositories as i do. (and id be useful for me too!)

I also agree that are lots of things to do. maybe more important. But I just wanted to share some pointers that I always thought could be very useful and easy to do now that this matter is argued :)

what I would love

Posted Jan 29, 2011 18:17 UTC (Sat) by pboddie (guest, #50784) [Link]

I meant all kind of useful path or env variables, not just conf files. And yeah its a matter of the policy, but thats what they are trying to sort it out in this article, isnt it? :)

It would be very useful to be able to query something and get this kind of information, although gathering it together in the first place would be some kind of documentation problem: the code surely knows which environment variables and configuration files to read, but this information has to make it out of the code and into some other kind of description.

I only refer to policy because this provides the basis of any decision about where programs should put their configuration files, for example. But then again, having tried to get packages into Debian and running into policy errors (and being told to read the somewhat unfocused set of manuals), I can totally agree with the idea of having tools that make such information more available to people who might not be familiar with "where things go".

I also agree that are lots of things to do. maybe more important. But I just wanted to share some pointers that I always thought could be very useful and easy to do now that this matter is argued :)

I think it's great to have these kinds of discussions, though. :-) And nicer front-ends to repositories - I can imagine something like a Web shop (but without any financial transactions, of course) - is probably something on my long list of things to play with... if I ever get the time.

Debian upgrading

Posted Jan 27, 2011 4:12 UTC (Thu) by jjs (guest, #10315) [Link]

@rahulsundaram -
You stated:
"I can install it immediately and in Linux, one usually has to upgrade their entire distribution. In Linux, there is no fundamental differentiation between system packages and even leaf applications. This is a problem that one has to solve and the current methods are awfully weak. "

How is upgrading Firefox and depenndecies "upgrading their entire distribution"? Debian will only upgrade packages that have changed or bring in new packages that are needed for the system.

Note that debian puts xulrunner in a separate package so that it is avialable for various packages.

Also, when I put iceweasel/experimental as an example - just upgradeing iceweasel to the latest version in whataver Debian distro you're using (stable, testing, unstable) is "apt-get -f install iceweasel".

Linux libraries

Posted Jan 27, 2011 4:15 UTC (Thu) by jjs (guest, #10315) [Link] (3 responses)

@rahulsundaram -
Also, Debian can have multiple versions of a package installed (xulrunner1.8, xulrunner1.9) to allow different packaages to use different versions of a library (unlike Windows, Linux libraries have different versions / sonames).

Linux libraries

Posted Jan 27, 2011 12:51 UTC (Thu) by pboddie (guest, #50784) [Link] (2 responses)

@rahulsundaram

May I recommend the "Reply to this comment" button? That way, rahulsundaram can follow your responses conveniently using the user tools and you don't end up shattering threads into many pieces.

Comment buttons

Posted Jan 27, 2011 14:49 UTC (Thu) by jjs (guest, #10315) [Link] (1 responses)

Because for some reason I don't get the "reply to" button? (just figured out I can hit the link button on a comment and get the ability to post a comment to the comment - not certain why I get a "Post a Comment" button under the article, but not a "Reply To" button under all the comments - possible bug in the system, or possible bug in my browser?)

Comment buttons

Posted Jan 27, 2011 16:44 UTC (Thu) by bronson (subscriber, #4806) [Link]

It's a long-standing intermittent LWN bug. I see it every few months. Just hit reload and the reply button appears.