|
|
Subscribe / Log in / New account

The universal application distribution mechanism?

By Jonathan Corbet
June 15, 2016
The announcement from Canonical that its "Snap" application package tools have been ported to a set of other distributions has generated some waves in the technical press. But developers for one distribution, at least, are not uniformly displaying happiness about this move on Canonical's part. Snap might yet succeed in unifying and simplifying application package creation and installation for multiple distributions, but it might also prove to be just another move in the interminable package-format wars.

Snap is an attempt to address a number of the problems that have emerged in the traditional Linux packaging ecosystem over the years. To get around dependency issues, Snap packages ("snaps") generally bundle almost all of the software they need to run. Security concerns are addressed by running Snap-packaged applications in a restricted directory space, confined by the AppArmor security module, and further constrained by a set of seccomp filters. There is an "app store" mechanism for distributing and installing snaps, and a set of tools intended to make it easy for application developers to create their own snaps. Snap includes a transactional update mechanism intended to make application upgrades safe and painless.

Application developers have often wished that they could build a single package that would run across all Linux distributions. The nearly moribund Linux Standard Base (LSB) effort tried for many years to create an environment where that was possible, but the application ecosystem that the LSB developers were hoping for never materialized. Now Snap is being promoted as the true answer to the Linux application distribution problem, so it is not surprising that it has garnered a certain amount of interest.

The Canonical announcement claims that snaps now work on Arch Linux and Fedora, along with the expected Ubuntu variants. Support for CentOS, Gentoo, Linux Mint, openSUSE, OpenWrt, and RHEL is said to be in the works. It is too soon to say how well it will work on all of those distributions, since they don't all support all of the features needed by Snap. In particular, most distributions lack Ubuntu's patched version of AppArmor; the word is that containment is simply not provided on those distributions. The instructions for Fedora state that, for now, SELinux must be disabled — a little discouraging for a system that is meant, among other things, to be a security technology.

Some of the early articles suggested that Fedora had actively worked with Canonical to do the port; that came as a bit of a surprise to the Fedora community, which quickly verified that Canonical had never actually said any such thing. In general, Snap seems to have gotten a bit of a chilly reception in the Fedora community; see, for example, this post from GNOME developer and Fedora contributor Michael Catanzaro:

What's going on is that Canonical beat us to market in development... and now their marketing folks have beat us in marketing, too. We of course have zero plans to adopt Snappy in Fedora, and in fact multiple Fedora developers are working on a competing solution, Flatpak (formerly xdg-app), which is also being adopted by GNOME and Endless.

To find more information on Flatpak, see this article and this one for LWN's coverage back when Flatpak was still called "xdg-app." Flatpak has similar goals to Snap's, but, naturally, it differs significantly in the details.

Both Snap and Flatpak are designed around bundling applications with their dependencies, but Flatpak tries to limit that bundling by having packages depend on "runtimes" containing many of the basic dependencies. That should make the packages smaller and, perhaps, address some of the security concerns associated with bundling of libraries. The cost, of course, is that Flatpaks have dependencies associated with them. The idea is that the runtimes will be small in number and will be the same on all distributions, so one Flatpak should still run everywhere.

The Flatpak sandbox makes heavy use of namespaces — a difference from Snap, which generally avoids namespaces. As with Snap, seccomp is used to disable system calls that are deemed to be unnecessary. There are plans to use SELinux to further confine each sandbox, but that appears to be a future item, as does the "portals" concept that gives the user control over what the application can access outside of the sandbox. The lack of portals, in particular, means that Flatpaks still run with the sandbox disabled for now.

It seems unlikely that the Fedora and GNOME developers will simply set Flatpak aside and go with Snap instead. Michael went as far as to suggest that Fedora might refuse to allow the Snap package entirely, since it might undercut the Flatpak project; he was quickly informed, from a few directions, that "does not compete inconveniently with a Fedora project" is not on the list of criteria for inclusion in Fedora. But Fedora has invested significantly in this project — Fedora 24 will include Flatpak support in the GNOME Software application — and seems certain to continue to do so.

What direction that effort takes will be interesting to watch; the conversation made it clear that there are still a lot of unanswered questions about how Flatpak will be integrated into Fedora. What happens if an upstream project starts distributing Flatpaks that duplicate a Fedora package; should that package then be removed from the repository? What happens if the upstream then stops distributing its own Flatpaks, or fails to address a security problem? And so on.

To further complicate matters, distributors are not just facing a choice between two competing next-generation package formats. AppImage is yet another attempt to solve the same problem; it already runs on a wide range of distributions. There are also the various container formats, such as Docker and rkt, in the mix; they, too, are intended to serve as a mechanism for distributing applications. Distribution of full virtual machines is also not all that uncommon. As with traditional package formats, the Linux community does not suffer from a dearth of choices.

The porting of Snap to other distributions seems like welcome news for the community as a whole; it represents an attempt to solve some stubborn problems. But, contrary to some current headlines, it still does not appear that Linux will have a single, universal application-distribution mechanism in the near future. Instead, it rather looks like a confusion of competing formats and mechanisms, perhaps not entirely dissimilar to what we have now. Over time, it may well be that a relatively small number of formats will dominate and life will be simpler. But one should not expect that to happen this year, and it is not going to happen without some fights.


to post comments

The universal application distribution mechanism?

Posted Jun 16, 2016 4:22 UTC (Thu) by vwduder (subscriber, #58547) [Link] (1 responses)

This does seem to be missing an important aspect of the Snap system. The only way to distribute an app is via the Canonical store, because the server is is non-free software and not available to run for yourself.

The universal application distribution mechanism?

Posted Jun 16, 2016 6:32 UTC (Thu) by zyga (subscriber, #81533) [Link]

Technically you can distribute your snap on any dumb http server and simply download and install a snap from a file using "snap install ./path/to/some/file.snap"

The universal software distribution mechanism?

Posted Jun 16, 2016 6:07 UTC (Thu) by pabs (subscriber, #43278) [Link] (8 responses)

I wonder if any of these formats are capable of being built directly from distro binaries?

The universal software distribution mechanism?

Posted Jun 16, 2016 6:31 UTC (Thu) by zyga (subscriber, #81533) [Link] (7 responses)

Snaps can be built from distribution packages (binaries) using snapcraft (https://github.com/ubuntu-core/snapcraft).

The universal software distribution mechanism?

Posted Jun 16, 2016 7:07 UTC (Thu) by halla (subscriber, #14185) [Link] (1 responses)

Yes, and that's what made it so amazing for me: I saw that all dependencies were pulled in from the ubuntu repos when I made the Krita snap, and still the result runs fine on other distributions.

That said, the article says "AppImage runs on a wide range of distributions". In my experience, after some noodling with opengl libraries and dri and stuff, it runs everywhere. Without any runtime needed, which is a big advantage. The only dependency is fuse, but even that is easy to circumvent by manually mounting the image and starting the application.

The reason there are two appimages of Krita for Linux, is that I messed up: I needed a version of gcc that can do OpenMP, and so I built that on my CentOS vm. The result is an appimage with a glibc that doesn't run on CentOS 6.5 or Ubuntu 12.04 anymore -- at least, I think that's what's up, but I haven't had time to figure it out.

So, of the two or three technologies I've tried, klik was clearly too early and too complicated to use, and disks were too small back then. AppImage needs quite a bit of noodling before the recipe works, once it works, it's easy to automate. You do need to build a bunch of dependencies manually, but that also means those deps can be configured and patched to fit my needs. Snap is pretty easy to use, and the Ubuntu appstore pretty easy to upload to, but re-using Ubuntu-built deps might not be the best thing. For instance, we'd really like to patch Qt 5.6.0 to fix some bugs.

So, we're full-circle: it's amazing that it works cross-distro, with the Ubuntu-built deps, but tighter control over what gets packaged is actually also nice.

The universal software distribution mechanism?

Posted Jun 16, 2016 8:16 UTC (Thu) by zyga (subscriber, #81533) [Link]

Just my last 0.02 on this, you can use binaries from the distribution... or you can build your own Qt and put it in the snap as a part. You can feely mix and match this and as long as the result works for you, it will work for everyone.

The universal software distribution mechanism?

Posted Jun 16, 2016 8:39 UTC (Thu) by pabs (subscriber, #43278) [Link] (4 responses)

Got an example of this?

The universal software distribution mechanism?

Posted Jun 16, 2016 9:04 UTC (Thu) by pabs (subscriber, #43278) [Link] (3 responses)

Looking at the tutorial it seems trivial for distros to automatically rebuild every app they contain into a snap package. snap.debian.org here we come!

https://github.com/ubuntu-core/snapcraft/blob/master/docs...

The universal software distribution mechanism?

Posted Jun 16, 2016 21:10 UTC (Thu) by epa (subscriber, #39769) [Link] (2 responses)

I wonder if you can also apply the reverse transformation: given a snap package factor out the libraries which already exist on the system, leaving a smaller residual package to install. It would then be possible to update libraries the application uses (for security fixes, and so on) without having to get a new version of the snap package (as when upstream is unresponsive, or the only new version available includes other changes you don't want).

The universal software distribution mechanism?

Posted Jun 17, 2016 21:57 UTC (Fri) by pgoetz (subscriber, #4931) [Link] (1 responses)

Doesn't this kind of defeat the purpose of having a sandboxed application? Re-inserting dependence on external libraries which can be updated and consequently break the application is exactly the thing that Appimage is designed to prevent.

The universal software distribution mechanism?

Posted Jun 19, 2016 15:23 UTC (Sun) by lsl (subscriber, #86508) [Link]

If the bundled dependencies are the reason the program is broken, replacing them wth different versions is *fixing* the application, not breaking it.

The universal application distribution mechanism?

Posted Jun 16, 2016 8:55 UTC (Thu) by mjthayer (guest, #39183) [Link] (5 responses)

I am waiting for the day when bundled GUI libraries like Qt are able to fit in with the look, feel and style of the system version. Hopefully all the new interest in App packaging will push people to get this working better!

The universal application distribution mechanism?

Posted Jun 17, 2016 7:03 UTC (Fri) by jospoortvliet (guest, #33164) [Link] (4 responses)

Would that be so hard? They would need read-only access to the user configuration in $home...

The universal application distribution mechanism?

Posted Jun 17, 2016 7:49 UTC (Fri) by mjthayer (guest, #39183) [Link] (3 responses)

The problem to my knowledge is that the look and feel is generally encoded as plug-in code which only works with a particular build of the GUI libraries. I am pretty sure this applies to Qt, not sure about Gtk. Of course, one can build the most common plug-ins along with the bundled GUI library, try to read the system configuration and hope to be lucky, but that is not a reliable solution.

If I have got something wrong here I would love to hear about it.

The universal application distribution mechanism?

Posted Jun 17, 2016 15:28 UTC (Fri) by flussence (guest, #85566) [Link] (2 responses)

Someone correct me if I'm wrong, but doesn't Qt5 do away with most of the theme plugin headaches by supporting some form of CSS directly?

(Although maybe not, since for all the buzz about Gtk3's theming API I've heard precisely nothing of people trying to do the same with Qt...)

The universal application distribution mechanism?

Posted Jun 17, 2016 16:00 UTC (Fri) by halla (subscriber, #14185) [Link] (1 responses)

That's because CSS-based theming was first introduced in Qt 3 (well, maybe Qt 2, it's a long time ago). It turns out to be a pretty crappy way to do theming, but it's still occasionally useful, for instance to give a widget a contrasting color as a warning or something like that. As an idea, it's old-hat, as a technique, it's been superseded.

The universal application distribution mechanism?

Posted Jun 17, 2016 22:26 UTC (Fri) by flussence (guest, #85566) [Link]

It may be an ugly hack, but there's a significant chunk of users that really like being able to mess with what's on their screen using only a text and image editor. Though I guess that's what the GTK+2 engine in Qt is for; people can just continue to use that.

The universal application distribution mechanism?

Posted Jun 16, 2016 9:53 UTC (Thu) by johannbg (guest, #65743) [Link] (22 responses)

"We of course have zero plans to adopt Snappy in Fedora,"

It's good to see the what gets shipped and used in the distribution get's dictated and decided by the ( Red Hat ) desktop team without any prior discussion in an alleged community distribution so they can force their pet project upon the whole ( Fedora) world for early adoption.

Somethings never change...

The universal application distribution mechanism?

Posted Jun 16, 2016 11:19 UTC (Thu) by pizza (subscriber, #46) [Link] (18 responses)

Yeah, how dare they make not plans to adopt something immediately after Canonical issued a press release.

The universal application distribution mechanism?

Posted Jun 16, 2016 12:56 UTC (Thu) by johannbg (guest, #65743) [Link] (15 responses)

?

It's more like how dare they to make a distribution wide decisions without consulting the community first out of their share arrogance and fear of healthy competition . If individuals or group like to package and implement snappy into Fedora ( or any other container solution ) there should be nothing preventing that ( unless there is some form of licensing issue ) then it's up to the sub community if they want to use that ( or any other containerized solution or just stick with the distributions packaging format ) instead of flatpak. So Red Hat's workstation product might just use flatpak while the rest of the community could/would use snappy or something else.

The universal application distribution mechanism?

Posted Jun 16, 2016 15:43 UTC (Thu) by pizza (subscriber, #46) [Link] (14 responses)

> It's more like how dare they to make a distribution wide decisions without consulting the community first out of their share arrogance and fear of healthy competition.

It's hardly "arrogant" to say that the Fedora Project has no plans to adopt Snappy when that is the abject truth. Arrogant would be claiming anything otherwise when there had been no discussion about it whatsoever. Also arrogant, I might add, was Canonical claiming to support Fedora (Step one: disabling system-wide security -- and "we'll fix that later") and implying that it was a collaborative effort without actually saying anything on fedora-devel before publishing their press release.

Meanwhile, The Fedora project leaders have been quite adamant in saying that they're not going to preclude anyone from packaging software however they want -- And yes, that includes using snappy or whatever.

So, please, continue to misrepresent what is on a public mailing list for all to see.

The universal application distribution mechanism?

Posted Jun 16, 2016 15:54 UTC (Thu) by johannbg (guest, #65743) [Link] (12 responses)

"We of course have zero plans to adopt Snappy in Fedora," by all means point me to the public discussion where they came to that conclusions in Fedora.

The universal application distribution mechanism?

Posted Jun 16, 2016 17:15 UTC (Thu) by pizza (subscriber, #46) [Link] (11 responses)

The second reply to the "hey, what's up with this snap thing" message was from Michael Catanazaro, which contains the "zero plans" quote. And I further quote:

"We have not considered, and need to discuss, whether to allow that snapcore package into Fedora proper .." [1]

In order for there to have been plans made to adopt Snappy, there would have had to have been some sort of discussion prior to Canonical dropping their press release bomb, and there was one. Hence the "zero plans to support Snappy" So no, I can't link to a discussion that didn't happen. But what I can do is link to the discussion that has transpired since the bomb was dropped:

[1] https://lists.fedoraproject.org/archives/list/devel@lists...

The universal application distribution mechanism?

Posted Jun 16, 2016 17:48 UTC (Thu) by johannbg (guest, #65743) [Link] (10 responses)

Interesting thread which touches upon what I mentioned earlier.

"Background info: In the Workstation working group, we're currently planning to allow replacing RPM packages for graphical apps with Flatpaks. We're also planning to remove Fedora packages for selected apps that are offered as Flatpaks by upstream."

The above statement contradict with what you said earlier about "The Fedora project leaders have been quite adamant in saying that they're not going to preclude anyone from packaging software however they want " Atleast Red Hat's desktop team did not get the memo about them not having any saying in what gets packaged in the distribution and what gets not.

There are more interesting bits in that thread which were only briefly mentioned here on the article but directly relates to fragmentation of appmarket and can have significant consequence for distribution as an result of that and that is what if upstream decides it will only support it's application in a single appmarket ( which at this point could either be flatpak or snappy or even something else) and how the system will handled the same app being install from different sources ( appmarket vs different appmarket or appmarket vs distribution package etc )

The universal application distribution mechanism?

Posted Jun 16, 2016 18:54 UTC (Thu) by pizza (subscriber, #46) [Link] (9 responses)

>"Background info: In the Workstation working group, we're currently planning to allow replacing RPM packages for graphical apps with Flatpaks. We're also planning to remove Fedora packages for selected apps that are offered as Flatpaks by upstream."

> The above statement contradict with what you said earlier about "The Fedora project leaders have been quite adamant in saying that they're not going to preclude anyone from packaging software however they want " Atleast Red Hat's desktop team did not get the memo about them not having any saying in what gets packaged in the distribution and what gets not.

A followup written 30 minutes later (Michael's very next email BTW) further elaborated: [1]

"There's no plan to systematically go around removing Fedora packages in favor of Flatpaks; rather, we plan to do this on a case-by-case basis at the request of upstreams that have developed Flatpaks and want those Flatpaks to be available in Fedora. Package maintainers would be allowed to dispute the change, again on a case-by-case basis, but I don't expect that to happen often. We're also planning to allow third-
party RPMs to replace Fedora-provided RPMs following the same procedure."

The conversation continues through several more emails. Another very closely-related one is [2].

"Certainly we're not going to come along and try to delete packages over the maintainers' objections. In general, I expect package maintainers would be deciding whether or not to make the switch [...]

Other parts of the discussion point out that behind-the-scenes work to enable Flatpaks have also made it easier to simultaneously support other software delivery mechanisms; Docker was specifically mentioned but Snappy would benefit too. Far from attempting to preclude various packaging mechanisms, Fedora is clearly trying to enable more options for its users.

(Incidently, the discussion also goes to show its readers that Fedora is not some monolithic entity where everyone marches in lock-step to someone's orders. There are lots of diverse interests represented, and the discussion continues..)

[1] https://lists.fedoraproject.org/archives/users/2069546229...
[2] https://lists.fedoraproject.org/archives/users/2069546229...

The universal application distribution mechanism?

Posted Jun 16, 2016 19:11 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link] (6 responses)

It is also worth nothing that Michael Catanazaro works for Igalia and is a GNOME developer on Epiphany and WebKitGTK so his perspectives tend to be naturally more aligned with GNOME. He also is fairly active in Fedora which uses Firefox as the default browser for example. So there is awareness of the differences in approach.

> Far from attempting to preclude various packaging mechanisms, Fedora is clearly trying to enable more options for its users.

This is worth emphasizing. There is some ongoing work on a proposal at

https://fedoraproject.org/wiki/Workstation/Third_party_so...

By the time, it gets hashed out, we will have a better understanding of how things like Flatpak needs to be handled.

There are several others which are related including

https://fedoraproject.org/wiki/Workstation/AtomicWorkstation
https://fedoraproject.org/wiki/Changes/Layered_Docker_Ima...

All of this represents a general trend towards being more flexible about how software is delivered to users.

The universal application distribution mechanism?

Posted Jun 17, 2016 1:55 UTC (Fri) by johannbg (guest, #65743) [Link] (5 responses)

That third party proposal is what the board has rejected on numerous occasion since it goes against one of the project foundation but the workstation group just wont accept that decision and let it go right?

The universal application distribution mechanism?

Posted Jun 17, 2016 2:26 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

I don't see any record of any such rejection. Do you?

The universal application distribution mechanism?

Posted Jun 19, 2016 17:40 UTC (Sun) by lsl (subscriber, #86508) [Link] (1 responses)

That not-yet-a-proposal is clearly intended to cover proprietary software (confirmed in countless mails on -devel). On every occasion the board was asked to "clarify" the compatibility of proprietary software with the community foundations, the answer was very clear. The "Workstation" WG reacts by changing some technicalities and pushing basically the same thing again.

How long is this farce going to continue?

Now that the old board got demoted(*), it's apparently seen as the perfect opportunity to get it through this time.

(*) replaced by the supposedly more "agile" Fedora Council
https://fedoraproject.org/wiki/Council

The universal application distribution mechanism?

Posted Jun 20, 2016 13:54 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]

> The "Workstation" WG reacts by changing some technicalities and pushing basically the same thing again.

The Board never outright rejected any such proposal. The "technicalities" are what the board was concerned with. How we expose third party software is important.

> Now that the old board got demoted(*), it's apparently seen as the perfect opportunity to get it through this time.

Nonsense. Its the same set of members.

The universal application distribution mechanism?

Posted Jun 17, 2016 3:25 UTC (Fri) by mcatanzaro (subscriber, #93033) [Link] (1 responses)

We sent it to the Council for feedback:

https://fedorahosted.org/council/ticket/57

They do have the power to reject it... I'm not expecting that outcome, though.

We've already been permitted to have third-party repo metadata enabled by default, so long as the repos are disabled by default and users get a clear warning about third-party software before the repo can be enabled. This is the status quo. The goal is to make it easier for users to find all the software they need from the software center.

The universal application distribution mechanism?

Posted Jun 19, 2016 21:12 UTC (Sun) by johannbg (guest, #65743) [Link]

Arguably this is an wrong way going about things since the future seems to reside in applications being available through an online application market/store which has powerful search capabilities, with either those application being free of charge or at a cost but not distributed through repository's scattered on the internet which end users or the software centers need to add/enable and know about.

If there is one thing the poorly named flatpak and snap have done then that is to have spark significant interest in AppImage and I would not be surprise if AppImage gains such application market and an integration to eclipse che which could generate the image and upload to an market as an alternative to that docker junk of theirs in not to distant future ,that that will be the final outcome in the application wars in the linux ecosystem.

The universal application distribution mechanism?

Posted Jun 17, 2016 1:51 UTC (Fri) by johannbg (guest, #65743) [Link] (1 responses)

"A followup written 30 minutes later (Michael's very next email BTW) further elaborated: [1]"

So in other words he figured out 30 minutes later that all these bold statements about what they ( the workstation group ) had decided was not flying with the community so he started to retract those statements since it's not like "We of course have zero plans to adopt Snappy in Fedora" and "We're also planning to remove Fedora packages for selected apps that are offered as Flatpaks by upstream" are somehow "vague" or "unclear" statements now are they o_O

Just for curiosity sakes do you work for Red Hat or are part of or associated with Gnome community?

The universal application distribution mechanism?

Posted Jun 17, 2016 2:15 UTC (Fri) by pizza (subscriber, #46) [Link]

> So in other words he figured out 30 minutes later that all these bold statements about what they ( the workstation group ) had decided was not flying with the community so he started to retract those statements since it's not like "We of course have zero plans to adopt Snappy in Fedora" and "We're also planning to remove Fedora packages for selected apps that are offered as Flatpaks by upstream" are somehow "vague" or "unclear" statements now are they o_O

Or perhaps he was responding in good faith to someone who asked a question in good faith, without either party assuming malice on the other's part? (I might add that these were the second, third, and fourth messages posted on that thread...)

> Just for curiosity sakes do you work for Red Hat or are part of or associated with Gnome community?

Nope. Though while I registered relatively recently as a Fedora developer, I've had accounts on both RH's and GNOME's bugzillas for a very long time indeed. I've never been particularly "active" in either, beyond filing bug reports and sending in occasional patches. (Oh, in full disclosure I perused RH's job listings about a month ago)

I've been running RH distros as both primary desktops and servers since the RHL4.0/Colgate release.

I also greatly prefer Gnome3 over Gnome2 (even at the time of the oft-derided 3.0 release) although most of my desktop real estate is typically taken up by terminals and emacs windows.

The universal application distribution mechanism?

Posted Jun 16, 2016 17:46 UTC (Thu) by zyga (subscriber, #81533) [Link]

FYI: SELinux doesn't have to be disabled on Fedora 23. On Fedora 24 it just prevents making a *socket* in /run (something probably trivial to fix but I'm not a selinux expert).

The universal application distribution mechanism?

Posted Jun 16, 2016 15:38 UTC (Thu) by johannbg (guest, #65743) [Link] (1 responses)

Also pay attention to the most likely second stage scenario from Red Hat desktop team playbook that they will use to force adaption of flatpak and that is to remove any application which is available as an flatpak from the distribution which will ensure such application is no longer available for the community made products ( like other desktop environments ) and will ( again ) cause other desktop environments to play catch up with Gnome.

The universal application distribution mechanism?

Posted Jun 17, 2016 3:17 UTC (Fri) by mcatanzaro (subscriber, #93033) [Link]

We seriously had not expected that this would be controversial. Clearly that was wrong; it's gonna require further discussion.

The universal application distribution mechanism?

Posted Jun 17, 2016 3:12 UTC (Fri) by mcatanzaro (subscriber, #93033) [Link]

Johann, it's a shame that almost every time I see your name, it's next to some anti-Red Hat comment. "We of course have zero plans to adopt Snappy in Fedora" isn't even a decision, it's just a statement that we haven't made plans to do something. To my knowledge, we hadn't even discussed it before... it's not something that's ever come up, because Snaps were Ubuntu-specific, so why would we think about them?

Anyway, you'd have a stronger argument here if not for the fact that *I don't even work for Red Hat*. I've merely been participating in the *public* discussions that have been occurring for months now, on the desktop mailing list (which I presume you choose not to read) and on IRC. We hold regular meetings on Freenode that are announced in advance on the desktop list, and which are open to public participation. We post public minutes and logs, like this meeting we announced and held on Flatpak two months ago:

https://meetbot.fedoraproject.org/fedora-meeting/2016-04-...

Certainly Red Hat has tremendous influence on the development of Fedora, but making "distribution wide decisions without consulting the community first" seems far from the truth. We haven't even decided anything yet, other than to submit our proposal to the Fedora Council for feedback. After that, it will surely need to be revised -- some folks on devel@ have pointed out problems already, and raised unanswered questions. That sounds like feedback to me.

The universal application distribution mechanism?

Posted Jun 17, 2016 13:50 UTC (Fri) by anselm (subscriber, #2796) [Link] (1 responses)

At face value, “we have no plans …” is really a statement about the present, not the future – there are no plans now but tomorrow or a week or a month from now that might well change, possibly depending on what goes on in the rest of the world (if for example, hypothetically speaking, all of the Linux community except the Fedora project took to Snappy like a duck to water, it would obviously be silly for the Fedora project to keep refusing to have anything to do with it, in perpetuity). Then again it might not.

In the absence of more context, I wouldn't bet the farm on a particular interpretation, especially given that Snappy is a fairly new idea and the Fedora people might simply not have had time to decide what to do about it.

The universal application distribution mechanism?

Posted Jun 19, 2016 18:18 UTC (Sun) by lsl (subscriber, #86508) [Link]

> In the absence of more context, […]

Another mail from the same day provides some bits of that context:

> We have not considered, and need to discuss, whether to allow that snapcore package into Fedora proper; there's a strong argument to be made that we should accept all free software, but doing that could undercut our Flatpak effort. If popular upstreams start distributing snaps, then we'll probably have to support it, though.

(taken from 1465932737.4887.24.camel@gnome.org)

The universal application distribution mechanism?

Posted Jun 16, 2016 15:35 UTC (Thu) by civodul (guest, #58311) [Link] (16 responses)

The current trend of bundling an app and all its dependencies in a single blob as popularized by xdg-app, Docker, and now Snap, is concerning: those bundles are typically opaque (where's the Corresponding Source?), and are a security concern (how does one make a security update in all those "app bundles"?).

Sure, "traditional" distros can be "slow", which is an occasional source of frustration for upstream developers. But I very much agree with Kyle Keen's article that distro maintainers have the big picture and play in important role in doing integration.

As someone working on GNU Guix, I think that what we propose is a more sustainable way to address the shortcomings of "traditional" distros: better composability and more flexibility, while retaining the quality of integration of packages and the ability to quickly address security and other cross-cutting issues.

The universal application distribution mechanism?

Posted Jun 16, 2016 16:02 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

> where's the Corresponding Source?

So being the oddball I am, I currently run FreeBSD with a bunch of jails for my home "cloud" infrastructure. I'd like to move over to containers, but everything seems to just start with a large base and go from there and I'd like to keep things minimal (e.g., using a single PostgreSQL server rather than one per web service or whatever as most images I found tend to do).

To that end, I'm working towards using Buildroot to build my images (I also have steps for using Debian as a base for things like git-annex which just aren't going to be in Buildroot any time soon). The idea is to store the rootfs images in hidden git refs (refs/images/new/$name) and add git notes for information like the buildroot hash, the config file root used, git hash of the repo I'm using to build things, etc. Once an image is set for deployment, it is moved over to a name like refs/images/deployed/$name and once a new set of images is created, another ref is created (refs/images/deployment/$date) with contents of the deployed images and such.

This should make it easy to figure out what built what and getting back an old deployment just needs to read the right refs/images/deployment/$date and fetch the right git hashes for their rootfs images.

The universal application distribution mechanism?

Posted Jun 16, 2016 16:10 UTC (Thu) by ortalo (guest, #4654) [Link]

For me it seems that this is probably not a package format confrontation, but the first steps in a repo. war. Probably a closed source repo. by the way.
So... thanks for the Guix pointer, and do not forget to supply an open/free repo. with it. As easy and usable as those phone users have been used to access. And I am sure you can try to do it with funny security options too.
("Dear computer, I read that article about hackers in the local newspapers yesterday. Challenge 1 out of 100 downloaded binary program randomly please. And explain me what is the difference betwen a developper and a compiler by the way..." On the next day, you propose the user to learn about crypto. hashes and then we will have another chance to save the world. :-).

However, bonus question: (how) do you intend to convince other distributions to rely on your packaging?
(It once seemed to me that the key initial reason for starting a new distribution was to create a playground for a new packaging system.)

The universal application distribution mechanism?

Posted Jun 16, 2016 17:04 UTC (Thu) by halla (subscriber, #14185) [Link] (11 responses)

I don't give a fig about "opaque, where are the sources", because that's covered in krita/3rdparty/. All the sources are collected in one place, and checksummed. I think "security, what about updates!!!" is a red herring because users mostly just don't update.

And the slowness of distributions to package the latest version of a productivity application isn't "an occasional source of frustration for upstream developers" -- I can stand being frustrated. It's a permanent source of frustration for the people who need the latest version of an application like Krita or Blender to earn their bread with. This is not about me, as an application developer, it's about the users.

Nobody, and that's literally _nobody_ who uses Blender on Linux seriously, uses a distribution-packaged Blender. Blender is a really big open source project: it is bigger than KDE or Gnome in terms of budget or community reach. When it comes to Krita, helping users to be more productive is my number one goal. Because I am a free software fanatic. I want people to start using free software.

In really plain words: when it becomes so easy to use Krita on Linux that most existing Linux users simply cannot understand that all they need to do is download and make executable, it's finally getting easy enough that we might get people from other platforms to start using Krita. Heck, Krita has converted hundreds of people to Linux and free software.

It's for the existing Linux users that I'm thinking of tarballing the executable appimage and telling them to untar it -- making the appimage executable is beyond their understanding in a lot of case.

The universal application distribution mechanism?

Posted Jun 17, 2016 8:32 UTC (Fri) by civodul (guest, #58311) [Link] (10 responses)

> I think "security, what about updates!!!" is a red herring because users mostly just don't update.

I don't think so. Server admins obviously care about it. Desktop users who have, say, Debian with GNOME 3, get cute notifications when security updates are available and they just have to click a button to be "safe". The situation is different in an "app bundle" world.

As for the need for users to get the latest software versions---sure, this is obviously a goal everyone agrees on. But we should analyze the reasons why distros fail to ship the latest and greatest in a timely fashion, rather than throw the baby out with the bathwater.

I think one of the reasons is that binary distros that follow a stateful/imperative packaging paradigm (Debian, Fedora, etc.) have a hard time testing all the possible configurations (e.g., there could be ABI breaks at the user site, depending the the user's machine current state). This tends to favor conservatism.

Conversely, by construction, functional package managers like Guix and Nix guarantee reproducibility: if it "works" on the maintainer's machine, it'll work anywhere else; if it builds on the maintainer's machine, it'll build anywhere else. This allows maintainers to be much more daring, and thus to more closely follow upstream releases.

Functional package managers, like app bundles, capture the whole system stack---but they do so without sacrificing composability.

The universal application distribution mechanism?

Posted Jun 17, 2016 9:12 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (9 responses)

> As for the need for users to get the latest software versions---sure, this is obviously a goal everyone agrees on. But we should analyze the reasons why distros fail to ship the latest and greatest in a timely fashion, rather than throw the baby out with the bathwater.
They've been analyzed to death, resurrection and zombification already.

The main reason is "dependency hell". Application developers want to use fairly recent versions of dependencies _and_ they want to have consistent dependencies across platforms. And I really mean "consistent" - you never know if Python 2.7.8 on Ubuntu is broken because poopyheaded maintainers backported a bad patch, while Python 2.7.8 on CentOS is perfectly OK.

Thus the Flatpak/Snappy solution. Everything else has failed. And no, automated build systems do not help because you still have to do QA for each build (if you care about your users).

Distro packagers were supposed to help with this, but apparently it's simply not enough.

The universal application distribution mechanism?

Posted Jun 17, 2016 13:08 UTC (Fri) by civodul (guest, #58311) [Link] (7 responses)

> The main reason is "dependency hell". Application developers want to use fairly recent versions of dependencies _and_ they want to have consistent dependencies across platforms.

This is an area where distros with a single package name space have difficulties: for most packages, there can only be one single version available on the system at one point in time.

Functional package managers do not have this restriction: they can describe precisely the dependency graph of packages. If one package needs a specific version or variant of another package, that is perfectly fine and it does not interfere with other users of that dependency.

As an example, Guix defines 3 variants of webkitgtk. There are 4 packages that depend precisely on webkitgtk 2.4, 12 that depend on 2.12, and so on; they can all coexist on the same system. There may be complex dependency graphs, but no "hell".

The universal application distribution mechanism?

Posted Jun 17, 2016 14:15 UTC (Fri) by micka (subscriber, #38720) [Link] (2 responses)

I'm curious. What happens if you have two dependencies that depend on different versions of a third one ?

The universal application distribution mechanism?

Posted Jun 17, 2016 14:54 UTC (Fri) by civodul (guest, #58311) [Link] (1 responses)

> I'm curious. What happens if you have two dependencies that depend on different versions of a third one ?

It depends. If a program ends being linked (indirectly) against two different versions of the same library, it may break badly at run time, no wonder.

However, it is possible to identify such situations beforehand by analyzing the dependency graph of the package, and the package's test suite would hopefully catch it anyway. In practice, we've never stumbled upon such problems.

The universal application distribution mechanism?

Posted Jun 17, 2016 17:20 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

The problem I usually see is due to unmangled bundling of dependencies causing symbol collisions. See Python wrappers which build their own versions of the C library.

The universal application distribution mechanism?

Posted Jun 17, 2016 17:19 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (1 responses)

Fedora has 3 builds of WebKitGTK too: WebKit1 with GTK2, another with GTK3, and the WebKit2 build. WebKitGTK is a special flower for this; have another example?

The universal application distribution mechanism?

Posted Jun 18, 2016 17:30 UTC (Sat) by civodul (guest, #58311) [Link]

> Fedora has 3 builds of WebKitGTK too: WebKit1 with GTK2, another with GTK3, and the WebKit2 build. WebKitGTK is a special flower for this; have another example?

With RPM, dpkg, and co., this is possible only by carefully choosing package names and versions; the package name and version effectively encode details about the specific variant.

Conversely, Guix does not refer to packages by name but by _identity_. Two different package objects defined at the Scheme level map to two different directories on the file system.

The universal application distribution mechanism?

Posted Jun 17, 2016 20:18 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

The singular namespace is not really a problem, you can easily create forked versions of libraries. It's the fact that you'd need to backport the libraries with their transitive dependencies.

"Functional" package managers make it easy to do - they essentially offer an easy way to build Flatpaks from source. That works, but personally I don't want to spend time compiling the code.

The universal application distribution mechanism?

Posted Jun 23, 2016 6:46 UTC (Thu) by callegar (guest, #16148) [Link]

Which is funny, because libraries are versioned and different versions of the same library can coexist on the system (with python dependencies being the notable exception as python packages are not versioned). So, there is no reason why a distro cannot ship, say, debs with a QT 5.6.1-1 runtime side to side to debs with a 5.5 one (-dev packages may be a different thing).

The universal application distribution mechanism?

Posted Jun 17, 2016 22:10 UTC (Fri) by pgoetz (subscriber, #4931) [Link]

Add the non secular Appimage to your list as well. Barring technical superiority of some solution, I lean towards supporting Appimage just so that it doesn't turn into another Canonical vs. Red Hat war of idiocy which makes life hell for users and (most particularly) sys admins in heterogeneous environments.

The universal application distribution mechanism?

Posted Jun 17, 2016 23:31 UTC (Fri) by joey (guest, #328) [Link] (1 responses)

Guix and Nix's common package format is much more of a "next-generation package format" than this app stuff. I don't think LWN has really covered them in any detail unfortunately.

The universal application distribution mechanism?

Posted Jun 23, 2016 18:04 UTC (Thu) by hitmark (guest, #34609) [Link]

Gobolinux may also be worth a look.

The universal application distribution mechanism?

Posted Jun 18, 2016 7:48 UTC (Sat) by pabs (subscriber, #43278) [Link]

The universal application distribution mechanism?

Posted Jun 24, 2016 19:05 UTC (Fri) by mina86 (guest, #68442) [Link] (6 responses)

Here's a stupid idea: compile binary statically.

The universal application distribution mechanism?

Posted Jun 24, 2016 19:13 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (5 responses)

How does that handle sandboxing, deduplication, delta updates?

The universal application distribution mechanism?

Posted Jun 27, 2016 15:01 UTC (Mon) by mina86 (guest, #68442) [Link] (4 responses)

Sandboxing is separate issue. Do one job and do it right. If you want to sandbox applications, come up with tool to do exactly that.

For de-duplication, you mean like what Debian or Red Hat packages already provide you? If you want to ship your programs with all dependencies, compiling statically does that. If you want to share dependencies, that’s what Linux distributions are already doing.

And debs and rpms already provide delta updates, so no need for new packaging format.

The universal application distribution mechanism?

Posted Jun 27, 2016 15:27 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

> Sandboxing is separate issue. Do one job and do it right

Sandboxing and file level deduplication are ways to mitigate the problems that can be caused by traditional solutions like static linking. They are directly related. You solution to that: Handwaving.

The universal application distribution mechanism?

Posted Jun 27, 2016 15:52 UTC (Mon) by mina86 (guest, #68442) [Link] (1 responses)

De-duplication is already provided by existing packaging systems.

Sandboxing is a matter of configuring what environment (namespaces, security managers etc.) the application is run in. It has nothing to do with how the application is packaged.

This whole thing seems to me like reinventing a wheel with some added features which could very well be created as separate to the wheel.

The universal application distribution mechanism?

Posted Jun 27, 2016 16:05 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]

> De-duplication is already provided by existing packaging systems.

Feel free to explain how statically linked packages can be deduplicated by existing packaging systems.

> This whole thing seems to me like reinventing a wheel with some added features which could very well be created as separate to the wheel.

I look forward to your solution to that.

The universal application distribution mechanism?

Posted Jun 27, 2016 16:34 UTC (Mon) by halla (subscriber, #14185) [Link]

" If you want to ship your programs with all dependencies, compiling statically does that."

I'd be interested to see you do a statically linked version of Krita that runs everywhere and has no loss of functionality compared to the appimage.

I think you will find it impossible to create one, but good luck.

What goes around comes around, how about we do it right this time?

Posted Jun 27, 2016 17:07 UTC (Mon) by philterry (guest, #28540) [Link]

I'll just leave this here http://www.tendra.org/


Copyright © 2016, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds