Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World) [LWN.net]

Here's a PC World article on the old, insecure version of ownCloud shipped in Ubuntu 14.04 — and the difficulties in getting it updated or removed.

Ubuntu’s developers initially balked at this. Why, this isn’t the way the system works! The package is now locked-in for the stable release and shouldn’t have any major changes, even though it’s a fundamentally insecure piece of server software. Actually removing it would be highly unusual. They proposed that ownCloud should take over maintenance of the ownCloud packages in Ubuntu and keep them up-to-date. At the very least, it was ownCloud’s job to create an empty package and go through the bureaucratic process to push it out.

The writing is a little breathless, but there is a valid issue here; the software found in the more remote corners of distribution repositories may not be particularly well maintained.

to post comments

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 18:42 UTC (Fri) by josh (subscriber, #17465) [Link] (25 responses)

There's some software that just shouldn't live in long-term "stable" distributions. For instance, youtube-dl (a script to download videos from YouTube and many other sites) frequently needs updates to keep it working. Older versions start failing on popular sites if not updated. Putting youtube-dl in a stable distribution and expecting it to work for a couple of years doesn't make sense.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 18:54 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

It perfectly fine to have youtube-dl in a stable distribution but the problem is when a distribution wants to treat all the software with the same level of conservative update policy.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:21 UTC (Fri) by josh (subscriber, #17465) [Link] (1 responses)

That's exactly what I meant by not putting it in a "stable" distribution. Putting it into some kind of auxiliary "volatile" or "backports" distribution, of newer software compiled to work with the stable core, makes more sense.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:28 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

I am somewhat confused by the use of word "distribution" there. Perhaps you mean different repositories? In any case, there is no problem in having a single repository with different updates policies. Fedora for example has

https://fedoraproject.org/wiki/Critical_path_package

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:43 UTC (Fri) by stefanor (subscriber, #32895) [Link]

Often one can find a minimal patch to keep something working, without the risk of causing regressions in behaviour.

When you can't, it is usually fairly straight-forward to get a SRU (stable release update) into Ubuntu, if it doesn't depend on newer versions of anything else. The SRU team understands the issues at play.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 8:23 UTC (Sat) by epa (subscriber, #39769) [Link]

And this shows why 'stable' is not the same thing as 'supported' or 'dependable'. A supported distribution of youtube-dl, if such a thing were to exist, would need regular updates to keep it working. There is this special biologist word we use for 'stable'. It is 'dead'. -- Jack Cohen

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 8:36 UTC (Sat) by drago01 (subscriber, #50715) [Link] (15 responses)

Only if you define stable as "set in stone no common sense allowed"

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 16:47 UTC (Sat) by drag (guest, #31333) [Link] (14 responses)

Wouldn't it be nice if it was possible in Linux to do things like 'Install _THIS_ version of Firefox' or '_THIS_ version of OpenCloud'?

That way Linux users could, you know, update the version of Gimp or OpenOffice they are using without having to reinstall the entire operating system (or upgrade it, which is often worse experience).

Of course it does if you completely ignore the traditional Linux software distribution model.

To bad very few distros are working on making installing and maintaining arbitrary software easy.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 17:34 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link] (3 responses)

Software collections does some of this

https://www.softwarecollections.org/en/

Also NixOS and Lennart's

http://0pointer.net/blog/revisiting-how-we-put-together-l...

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 19:22 UTC (Sat) by drag (guest, #31333) [Link] (2 responses)

Yes. Definately software collections is something I am interested in. However it's probably Lennart's solution that has the best long-term process.

Software should really be built and packaged by upstream in most cases, except for lower-level OS stuff. It would be, in my more perfect world, be the job of the distributions to help upstream do this and make it as easy as possible.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 13:12 UTC (Sun) by hkario (subscriber, #94864) [Link] (1 responses)

Lennart approach still requires /somebody/ to update the packages in base image when (not if) there are security issues in core libraries and then the users needs to know that he or she has to update the base image... - from security PoV that's unworkable

and if you don't care for security - just don't update at all, the software will continue to work

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 14:49 UTC (Sun) by raven667 (subscriber, #5198) [Link]

Yes, that is one of the reasons they were using btrfs, the distro can serialize delta updates of the whole file system for efficiently patching.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 0:04 UTC (Sun) by NightMonkey (subscriber, #23051) [Link] (9 responses)

Gentoo pretty much offers what you seem to be seeking. YMMV.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 12:32 UTC (Sun) by tterribe (guest, #66972) [Link] (2 responses)

Gentoo has a couple of problems.

1) They are way too aggressive about removing packages from the repository. This often leads to situations where upgrading/installing something pulls in a dependency that's so new it forces upgrades of 50 other packages (and then the process repeats), even though an older version of the package that's been removed from the tree would have avoided the problem.

2) The revdep-rebuild approach to reverse dependency tracking means that you can't always see how big of a mess this will be until you are in the middle of it.

However, if you're willing to pull old packages out of Gentoo's CVS and stick them in a local overlay, you can quite reasonably do what you want here. It just requires a bit more manual effort than it should.

You also run into the general problem that the dependency information in the packages is not always ideal, simply because very few people are testing very new packages in combination with very old ones, so sometimes things get missed (e.g., foo-2.3 requires >=libbar-1.2, but there were no versions of libbar older than 1.2 in the tree when foo-2.3 was packaged, so nobody remembered to update the rule).

But I will say that I have run into similar problems with every distro I've used, and Gentoo is the only one where I have been able to sit down and reasonably solve such problems, every time, in less time than it would have taken to just give up and re-image the OS with the latest Fedora release or whatever.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 23:11 UTC (Sun) by gerdesj (subscriber, #5446) [Link] (1 responses)

"2) The revdep-rebuild approach to reverse dependency tracking means that you can't always see how big of a mess this will be until you are in the middle of it."

Like all systems, Gentoo is evolving and improving. revdep-rebuild is pretty much a thing of the past (barring bugs). Portage does it automatically nowadays.

Even perlcleaner and python-updater are becoming increasingly unnecessary.

I'm seeing much less breakage these days that needs serious thought to fix up and my last visit to the "tinderbox" or another box for a binary package to get out of a proper mess was a pretty long time ago.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 21:55 UTC (Mon) by Wol (subscriber, #4433) [Link]

> Even perlcleaner and python-updater are becoming increasingly unnecessary.

so when you really need them, they're buggy :-(

Case in point, my latest emerge upgrade crashed because of missing perl modules or something. So I ran perl-cleaner, which crashed because of screwed-up dependencies.

Okay, following the instructions provided with the mess cleaned up the mess, but it wasn't nice :-( It would be nice if things "Just Worked (tm)" :-)

Oh - and one other nice enhancement I'd like to see to portage - which would get rid of a lot of need for --backtrack and the like - is an option that told it NOT to abort on dependency conflicts, but just to drop conflicting packages and carry on. The number of times I've had to manually select a couple of packages from the world list to upgrade, rerun the world list, manually run a few more packages from the list, rinse and repeat, and suddenly the conflict goes away. If I could just tell it to "emerge what you can", it'd probably sort itself in a couple of goes.

Cheers,
Wol

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 21:15 UTC (Sun) by rodgerd (guest, #58896) [Link] (5 responses)

For a tinkerer, perhaps. But it's completely useless for an audience who don't want to spend their lives in the guts of their distro.

Whereas someone running, say, Windows 7 (because they don't want to touch 8) can trivially use the latest Firefox and Python while remaining on Office 2010.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 23:14 UTC (Sun) by NightMonkey (subscriber, #23051) [Link] (4 responses)

Are you talking about Gentoo? The aspect of Gentoo I like is that complex stuff just works, and is well managed. I stay out of the 'guts' much more often than I do with RedHat and descendants in common use cases. (I consider 'guts' to include things like YUM plugins (yum-priorities, etc.), required community repositories (hmm... which one is good *today*?), for example.)

To you other well-made point, yeah, screw tinkerers. They suck, and are sooooo unprofessional. Nothing good comes from tinkering. ;)

You lost me on how the Windows 7/Firefox/Python is relevant here? There's lots of other non-technical business decisions inside Microsoft that underpin the idea of different 'versions' of Windows and critical core libraries.

Cheers.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 0:34 UTC (Mon) by rodgerd (guest, #58896) [Link] (3 responses)

And this whole comment - wilful ignorance, snottiness and all - is precisely why Linux is and will likely be forever irrelevant outside of embedded uses.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 21:58 UTC (Mon) by Wol (subscriber, #4433) [Link]

So why is it that the majority of my support grief comes from one of two sources (usually both together :-) namely

1) Windows
2) Users who ring for support, and half way through telling them what to do the do the exact opposite!

Cheers,
Wol

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 11, 2014 16:45 UTC (Tue) by justcs (guest, #92304) [Link] (1 responses)

Linux runs on a overwhelming majority of Supercomputers as well as desktops in the ISS.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 11, 2014 20:45 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

I think the conversion on the ISS was a more recent thing (though there is really only 1 of it), but IIRC, there were issues moving to Windows 7 (I'm pretty sure Vista was skipped) related to licensing. Here's a 2013 article that mentions stability concerns, so maybe licensing wasn't everything:

[1]http://www.extremetech.com/extreme/155392-international-s...

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 13:07 UTC (Sun) by hkario (subscriber, #94864) [Link]

even on Red Hat Enterprise Linux the NSS cryptographic library (the one Firefox uses) is rebased regularly - RHEL 5, released in 2007, has NSS from this year. Firefox is also updated to current ESR version (soon to be FF31)

sure, those are exceptions, but for some packages are necessary - either you have to commit to backport the fixes yourself or you're at the mercy of upstream, there's no 3rd option

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 22:54 UTC (Sun) by HenrikH (subscriber, #31152) [Link] (1 responses)

Well it works for Firefox, on my 14.04LTS I got v33 via normal apt-get update when it was released by Mozilla. So some packages get new versions even in LTS.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 1:47 UTC (Mon) by dlang (guest, #313) [Link]

That's because distros gave up backporting 'fixes' for firefox a few years ago, and even on the 'stable' versions they just ship the current firefox release.

It became too hard to identify the difference between fixes and features and to extract one from the other (and keep doing so as change happened)

The good news is that the Firefox team has gotten pretty good at avoiding breaking things as they upgrade (not perfect, but pretty good)

Most programs actually upgrade pretty well, but there are a handful of high profile ones (Unity and Gnome among them) that are fare more intrusive with their updates, and these cause the users to be reluctant to just upgrade everything.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 11, 2014 10:30 UTC (Tue) by ceplm (subscriber, #41334) [Link]

Well, some other long term distributions are actually, you know, maintained. LTS doesn’t “let’s call this old pile of turd LTS, and try to attract enterprise users on it”. People who actually do pay for Linux, expect the provider to do something more than publish couple of security patches distributor collects from the upstream. Which is the reason I am very happy there was never Fedora LTS.

Yes, I do work for Red Hat, and no I don’t think RHEL is the biggest achievement of humanity, and yes I do like CentOS (which is IMHO free LTS distribution done right, i.e., somebody else does actual work).

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:01 UTC (Fri) by wagerrard (guest, #87558) [Link] (4 responses)

What kind of policies/procedures exist in various distros to regularly purge packages that don't meet standards, and what are those standards?

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:18 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

There are a number of different policies in Fedora. The ones I am aware of that are relevant

https://fedoraproject.org/wiki/Deprecate_FTBFS_packages

https://fedoraproject.org/wiki/Retire_Orphaned_Packages

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:49 UTC (Fri) by stefanor (subscriber, #32895) [Link] (2 responses)

Ubuntu doesn't remove packages, post-release, because the release pocket (e.g. trusty, as opposed to the trusty-updates or trusty-security pockets) never changes post-release.

The only mechanism we have for removal, post-release, is replacing with a blank package, in the updates or security pocket.

In Debian, packages can be removed from stable at each point-release.
https://wiki.debian.org/ftpmaster_Removals
As usual in Debian, the decision to do this usually comes from the package maintainer, themselves. If the maintainer is AWOL, the project can do it via the QA team (which includes every developer)
https://qa.debian.org/howto-remove.html

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 15:02 UTC (Mon) by epa (subscriber, #39769) [Link] (1 responses)

What is the value of maintaining this property: "the release pocket never changes post-release"? Why not allow it to change?

After all, the set of available software does change as some packages are removed, and if you just replace something with an empty package that is equivalent to removing it for all practical purposes.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 21, 2014 10:17 UTC (Fri) by CameronNemo (guest, #94700) [Link]

I am pretty sure that if the package is removed from the repo, the users that have already installed it will still have it installed. If you replace it with a blank package, they will get the update and there will be no security issues left.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:30 UTC (Fri) by gwolf (subscriber, #14632) [Link] (27 responses)

Some comments have pointed out the need to keep it in backports/volatile — That's all fine, and that's what we do in Debian. And in Debian, we also backport as much as possible the security fixes to the versions in stable.

The *huge* difference is the meaning of the Ubuntu Universe: That's software available that /should work/, but is in no way supported. Universe is "community supported", and yes, there are the "Masters of the Universe", but it does not mean security updates will be prepared, coordinated or anything by Canonical.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:56 UTC (Fri) by stefanor (subscriber, #32895) [Link] (26 responses)

It's not that straightforward. There is software in universe that is supported by Canonical, and software in main that isn't really supported.

Run ubuntu-support-status, to find out the support state of a machine.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 20:03 UTC (Fri) by jspaleta (subscriber, #50639) [Link] (19 responses)

How does one see the state of Canonical support from outside a running Ubuntu system? Is there a full listing of what packages canonical formally supports (for whatever definition of support you are using here)?

I've never understood how potential support customers understand what they are actually potentially paying for with the supported packages scattered across repo boundaries like that.

-jef

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 20:06 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

Yes. The 'main' repository is officially supported. Community-supported packages go into the 'universe' repo.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 20:11 UTC (Fri) by jspaleta (subscriber, #50639) [Link] (1 responses)

Sounds like you are directly contradicting the statement I was relying to.

And in doing so, perhaps stressing the important point. I think Canonical's support story is overly confusing.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 1:40 UTC (Sat) by mdeslaur (subscriber, #55004) [Link]

How is it confusing? There a two completely separate package repositories. One is supported by Canonical, the other isn't.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 6:18 UTC (Sat) by xnox (subscriber, #63320) [Link]

"Yes. The 'main' repository is officially supported. Community-supported packages go into the 'universe' repo."

Whilst this is generally true "rule-of-thumb", it's more complex than that because of pleora reasons. Flavours can and do get LTS status, but are generally supported by said flavour developers. LTS status is a community process led by TechBoard. Binaries from same source package can be split between different components (main vs universe). Architecture one runs things on also matter (e.g. amd64 vs arm64). Things are "supported" but are not compiled from source (e.g. firmware & drivers in "restricted") etc.

And "supported" is a broad term - do you care about security vulnerability fixes only; or bug/functionality fixes as well; or new-hardware support? There are packages that have all,none of those properties and any combination in-between. And like any other company, given enough golden bricks one can get support contract for anything one wishes. Similarly the support relationships with OEMs & CPU designers is also quite different.

What I like about Ubuntu is that software is not excluded from the distribution simply because Canonical doesn't have engineers to support it. Instead there is plenty of opportunity for interested parties to contribute. Which also means, that even if one doesn't have support contract with Canonical and/or cannot negotiate one, there will be other people & companies happy to support you on Ubuntu.

Standard support contracts are detailed here http://www.ubuntu.com/management/ubuntu-advantage and there is "contact us" button to discuss any queries you have and/or negotiate additional things.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 20:14 UTC (Fri) by stefanor (subscriber, #32895) [Link] (14 responses)

There are support-status fields in the package lists. Improving the understandability of this has been on the cards for a long time, but there are always more important things to do...

As you can see from the quotes in the article, and the mailing list discussion, even some Ubuntu Developers don't understand support status :)

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 20:28 UTC (Fri) by jspaleta (subscriber, #50639) [Link] (13 responses)

Makes me wonder... how exactly are those priorities being assessed. This seems like this should have been a very important priority for Canonical's support business.

Because it seems to me... _focusing_ early on articulating your support story to your potential customers as concisely as possible would pay off over a long timescale in terms of revenue.

If noone is making a point of making sure that support story is crisp and clean. You don't want to lead customers on and give them the impression they are getting more support for their money than they are. That's a good way to burn bridges instead of building them. No idea if that's been happening, but it sure seems like the conditions are ripe for that.

Have fun.

-jef

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 1:42 UTC (Sat) by mdeslaur (subscriber, #55004) [Link] (12 responses)

It's completely crisp and clean. The "main" repository is supported by Canonical. The "universe" repository is not.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 1:46 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]

Didn't you read?

https://lwn.net/Articles/619504/

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 2:43 UTC (Sat) by jspaleta (subscriber, #50639) [Link] (10 responses)

Again that is in direct contradiction with the statement I was replying to.

But you know.. for the record.. can you point me to customer facing documentation that states that everything in main is supported by Canonical.. for future reference.

I've read the published terms for ubuntu advantage and it doesn't actually say which packages are supported and which aren't. If you can point me to a terms of service description, anything really, that is potential customer facing which says everything in main is supported and what that means that'd be great.

Oh and a definition of the different support levels too. That'd be groovy.

-jef

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 10:58 UTC (Sat) by johannbg (guest, #65743) [Link] (5 responses)

Anyone knows that if you require real enterprise support with real engineering backup there are just two companies capable of providing that and that's Red Hat and Suse.

If you require a distribution with "Long Term Support" then you deploy either Red Hat based distribution such as Centos or Scientific Linux or you deploy Debian LTS release directly and you simply call your local Linux IT company for support and counselling you dont use Canonical.

There are gazillion "cloud" providers out there providing wide variety of cloud based solution including but not limited to openstack.

Even Google does not list Ubuntu in it's distribution drop down for it's cloud [1].

For free it lists: Debian,CentOS,OpenSuse,FreeBSD,CoreOS etc. ( it does not list Ubuntu ) for paid it lists Windows,Red Hat and Suse, which just reflects what everybody already knows.

Bottom line Canonical is not doing anything special ( it may think it's doing something special when in reality it's not )and their so called support is no exception in that regard since you can get the same or better support from your local IT company as they provide.

https://cloud.google.com/products/calculator/

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 14:25 UTC (Sat) by gioele (subscriber, #61675) [Link] (3 responses)

> Anyone knows that if you require real enterprise support with real engineering backup there are just two companies capable of providing that and that's Red Hat and Suse.

RHEL does not include ownCloud in its repositories, so Red Hat will not support it. If you want ownCloud in your RHEL box you will have to use additional repositories.

This is not different from the Ubuntu situation: ownCloud is not present in the "main" repository, so, if you want it, you have to use additional repositories: either the "universe" repository or the repositories provided by ownCloud via openSUSE Build.

Should Canonical stop providing the unsupported-but-useful "universe" repository only because people, seeing the name "Canonical", may think that it is supported and maintained by Canonical?

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 15:04 UTC (Sat) by johannbg (guest, #65743) [Link] (1 responses)

"RHEL does not include ownCloud in its repositories, so Red Hat will not support it. If you want ownCloud in your RHEL box you will have to use additional repositories."

And if you use third party repository's you break your support contract and remember there always is a reason for companies not include said software in their repositories.

"Should Canonical stop providing the unsupported-but-useful "universe" repository only because people, seeing the name "Canonical", may think that it is supported and maintained by Canonical?"

The answer to that is yes since Canonical is misleading it's "customers" by providing it in the first place.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 16:31 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]

>And if you use third party repository's you break your support contract and remember there always is a reason for companies not include said software in their repositories.

Merely using a third party repository doesn't break any support contract. Red Hat merely won't commercially support packages from those repositories. Same goes for modified packages unless you can show that the problem occurs regardless of the modification.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 15:13 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

> Should Canonical stop providing the unsupported-but-useful "universe" repository only because people

I don't see it in the discussion here, but isn't it *on* by default? Maybe if there were an explicit action users would need to take to enable it, it wouldn't be such an issue (there may be (I don't know for sure) but I'd like that detail in the discussion here)?

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 1:37 UTC (Sun) by mjblenner (subscriber, #53463) [Link]

> Even Google does not list Ubuntu in it's distribution drop down for it's cloud [1].

Maybe, but it's an option:

https://cloud.google.com/compute/docs/operating-systems#u...

And I think you misunderstand 'paid'. I read it as saying you can't use those OS's without paying, not that you get more support.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 16:37 UTC (Sat) by mdeslaur (subscriber, #55004) [Link] (3 responses)

> But you know.. for the record.. can you point me to customer facing documentation that states that everything in main is supported by Canonical.. for future reference.

https://wiki.ubuntu.com/SecurityTeam/FAQ#Official%20Support

"Packages in main and restricted are supported by the Ubuntu Security team for the life of an Ubuntu release, while packages in universe and multiverse are supported by the Ubuntu community."

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 15:12 UTC (Sun) by niner (guest, #26151) [Link] (2 responses)

That's about security updates which is not the same as supported as in 'I have a problem and I can call my support contact and get some help'.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 1:48 UTC (Mon) by dlang (guest, #313) [Link] (1 responses)

It's pretty obvious that you haven't actually had to use the 'commercial' support.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 6:43 UTC (Mon) by niner (guest, #26151) [Link]

Yes, that's true :)

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 3:00 UTC (Sat) by jspaleta (subscriber, #50639) [Link] (2 responses)

Since the accuracy of your statement has been called into question by others... and the available wiki information seems to suggest that at least from a policy standpoint its suppose to be a clean split with Canonical not supporting anything in universe.

I feel I need to request that you provide an example of both a supported package in universe and an unsupported package in main? Any active release of Ubuntu that Canonical is currently willing to take paid support for.

Can you provide any such examples.. or on the record mailinglist discussion concerning the state of support the bleeds across the repo definitions contrary to wiki documentation.

-jef

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 4:56 UTC (Sat) by stefanor (subscriber, #32895) [Link] (1 responses)

Yeah, this really isn't well understood. Partly because the main/universe split still *mostly* aligns with support, just not entirely.

Some source packages are in main, but build some binaries that go to universe (e.g. nginx goes to main, nginx-extras goes to universe, but both get built from the same source package, at the same time). Other source packages are in main, because supported packages in main depend on them, but aren't intended to be supported, themselves.

There are 4 binary packages in trusty (an LTS release) amd64 main, that don't have Supported fields, and are therefore as (un)supported as a typical universe package:

curl -sq http://archive.ubuntu.com/ubuntu/dists/trusty/main/binary-amd64/Packages.bz2 | bzcat - | grep-dctrl -F Supported -s Package,Supported -v -r '.*'
Package: lib32go5

Package: lib32go5-dbg

Package: libdbus-cpp2

Package: libprocess-cpp1

The rest of main has variable support terms:

$ curl -sq http://archive.ubuntu.com/ubuntu/dists/trusty/main/binary-amd64/Packages.bz2 | bzcat - | grep-dctrl -F Supported -s Supported -r '.*' | sort | uniq -c
     70 Supported: 3y
   4376 Supported: 5y
   4116 Supported: 9m

Let's look at universe:

$ curl -sq http://archive.ubuntu.com/ubuntu/dists/trusty/universe/binary-amd64/Packages.bz2 | bzcat - | grep-dctrl -F Supported -s Supported -r '.*' | sort | uniq -c
    698 Supported: 3y
    959 Supported: 5y
   2040 Supported: 9m

Now, a Supported field doesn't mean that Canonical is the one providing the support, either. Some flavours offer LTS, without Canonical support. Others receive support from Canonical, as Canonical is selling support for the flavour to customers.

This is all, of course, confusing. And so there has been talk, for years, of merging universe into main, and exposing seed-based packagesets in the APT package-lists. See the archive-reorg discussions.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 16:43 UTC (Sat) by mdeslaur (subscriber, #55004) [Link]

> Partly because the main/universe split still *mostly* aligns with support, just not entirely.

If it doesn't align, it's a bug in the seeds that generate the supported field. The field isn't really use anymore anyway since we aligned desktop and server support periods to 5 years in Ubuntu 12.04 LTS.

It's pretty simple: if the binary package is in Main or Restricted, it's officially supported.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 16:35 UTC (Sat) by mdeslaur (subscriber, #55004) [Link] (2 responses)

> There is software in universe that is supported by Canonical, and software in main that isn't really supported.

First I've heard of this, and I'm the one doing security updates.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 9:27 UTC (Sun) by stefanor (subscriber, #32895) [Link] (1 responses)

Yeah, fair enough. You would know this better than I would.

Maybe I'm conflating commercial support with security support, in my mind.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 9, 2014 11:24 UTC (Sun) by dlang (guest, #313) [Link]

what do you consider commercial support?

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 19:39 UTC (Fri) by callegar (guest, #16148) [Link] (5 responses)

The issue described here is further complicated by the fact that the owncloud project actually ships updated packages for ubuntu via the Suse factory.

The point is that to assure maximum uniformity among the packages they provide for the different distributions, the ubuntu/debian packages that they ship are not compliant with the ubuntu/debian guidelines.

Now, I understand the desire of the owncloud project not to pick up the maintenance of the owncloud deb packages shipped by ubuntu, as this would basically mean supporting two different 'flavors' of deb packages, one not respecting the debian guidelines (but fully compatible with own cloud installations made from the tar.gz package) and one respecting the debian guidelines (for the ubuntu distribution).

It is even worse than that. Paradoxically, if the ubuntu maintaineres suddendly decided to start updating their owncloud packages, this would pose another issue. Users who have added the Suse factory repository to their apt lists and who are currently running the owncloud-shipped owncloud server packages might end up getting upgrades from the ubuntu version of owncloud (since the two have the same package names and versions and there is no coordination in the releases). Then, depending on the relative release pace they might start getting alternate upgrades from the two repos (unless they use the pinning features of apt). For instance, one may get X.Y-0 from Suse factory, then X.Y-2 from Ubuntu, then X.Y+1-0 from Suse then X.Y+1-1 from Ubuntu, etc. This would be highly problematic since the two versions are not fully compatible, because one follows the debian guidelines (e.g. location of config files, etc.) and the other does not. This may break installations.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 22:56 UTC (Sat) by jospoortvliet (guest, #33164) [Link] (4 responses)

First of all, we don't use suse factory - the openSUSE development tree is something we build packages against, but that is entirely separate from other packages. We use the open Build Service, which is a distribution agnostic build server - even our Windows packages are build on it.

Ideally, distributions like Debian would run their own OBS instance which could connect to ours so we would be able to work directly with Debian packagers (without having to leave our own infrastructure!) on bringing the packages up to spec. This provides great transparency and all the benefits of the current distribution model without the downsides caused by the balkanization the distributions have caused.

It is rather sad that all the technology is there but distributions prefer to stay on their islands and ignore the problem. And even bark at isv/open source projects occasionally for not helping the packagers. Wow, that surprises you? Get your act together and it becomes not only feasible but easy to collaborate!

I get why it is: nobody's itch needs scratching urgent enough. This harms isv's/foss projects writing software most and they focus on writing software, not starting projects in distributions to fix their problems.

Anyway. We did a bit of a write-ups on the issue on planet ownCloud, feel free to debate here or there. But please don't blame us for the fact that distributions fail to step over their own shadow and adopt (even just as an interface) technology that could improve this situation.

http://owncloud.org/blog/linux-distributions-and-open-sou...

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 8:15 UTC (Mon) by AdamW (subscriber, #48457) [Link] (3 responses)

As the Fedora and EPEL maintainer of OC:

I'm not sure technology is really the solution. It's not really difficult to follow how a given bit of software is built in the major distros. I mean, you can go to packages.debian.org and find all the necessary info on the Debian build, ditto packages.ubuntu.com for Ubuntu. For Fedora or SUSE or Mandriva or many other distros you can go check the package build out of an SCM. Packaging systems really aren't that complicated, I can follow the Debian build of OC fine though I've never built a Debian package in my life.

I think it's perfectly feasible for upstream and downstream to work together without any technology changes, all it takes is...for us to do it. I think the blog piece is an interesting one, but by considering mainly the Ubuntu situation it skews too negative. I'd say I and the Debian maintainer, for e.g., have a pretty healthy relationship with upstream - we hang out in #owncloud , we send patches upstream, we talk to you folks regularly, we update our packages pretty regularly (OK, it took me a bit to find time to bring EPEL up to speed, but it should be fine now).

I think it's a bit unfair to distros to say stuff like "distributions prefer to stay on their islands and ignore the problem". We each have our frustrations with each other, I can vent about sloppily bundled and under-documented dependencies for a while if you like ;) But just as we should recognize the realities of trying to maintain a large-scale project like OC maybe you can recognize that a major distro, especially one with well-established processes and tooling, is a big beast, it's an ocean liner, not a speedboat; distros are deeply, deeply tied to their package build systems and it's really not as simple as 'hey look, there's OBS over here, why not just build stuff with that instead?'

Distros can look finicky and old-school with our insistence on policies about source verification and bundled libraries and the integrity of the build chain and update policies and blah blah blah, but most of those things aren't just there to make things slower...

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 16, 2014 20:32 UTC (Sun) by jospoortvliet (guest, #33164) [Link] (2 responses)

First, thanks for the work you do. I am aware of the cross-distribution collaboration, esp between fedora and openSUSE.

I wouldn't suggest to build fedora with OBS either. What I do suggest is to have an OBS instance running where upstream can collaborate with downstream on building and maintaining packages. You thus share the work load, can always see what is going on and users can now trust BOTH their ISV and the distro people to protect them.

This model is technically possible and seems, to me, a very usable solution where both cakes are preserved. But I am not a packager and can't judge this entirely of course, so perhaps there are problems I am not aware off. Still, this model works great for software development (eg github) so I think it would be worth trying.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 17, 2014 22:23 UTC (Mon) by ceplm (subscriber, #41334) [Link] (1 responses)

I don't think working in the same building system achieves anything. It really doesn’t matter (and it shouldn’t matter) whether the code is build in OBS, Koji, or straight on my installed system. What does matter is willingness of upstream to cooperate with distributions packagers, accepting patches, and (especially) sticking to stable APIs. Unfortunately, concerning stable APIs, exactly people around PHP are usually the least to care about this. I have my own running into this issue at
http://thread.gmane.org/gmane.comp.kde.devel.owncloud/11452/

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 20, 2014 1:29 UTC (Thu) by raven667 (subscriber, #5198) [Link]

> It really doesn’t matter (and it shouldn’t matter) whether the code is build in OBS, Koji, or straight on my installed system.

While sure, you can get the same result no mater how you build it, and a distro has the resources to build whatever custom system they want, for third party packagers it would be nice if they only had to deal with one standard build system across the major systems rather than one per distro (plus package format differences). if an upstream is going to package their software they have to learn DEB and RPM, OBS and Koji, etc. which is asking a lot, a lot more than most upstreams are really able to give. OBS seems the most featureful and mature of the systems in common use so it seems to be a reasonable choice as a standard to build toward.

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 7, 2014 20:41 UTC (Fri) by alogghe (subscriber, #6661) [Link] (2 responses)

This raises many issues but the one I think is probably most important-

Packages like this are extremely high risk and should be in a completely different repository with very different guidelines.

This package has extremely personal user data that is expected to be directly shared over the open internet.

Why is this in the same archive, and treated in the same manner, as an mp3 tag renamer etc.?

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 8, 2014 23:39 UTC (Sat) by xxiao (guest, #9631) [Link]

Indeed different repos are needed in this case, e.g. put owncloud to a "volatile" repo together with similar projects.

redmine is another example, the version in ubuntu/debian is ages behind the newer ones, they should be put into a different repo instead, another example is drupal.

Then you do have the dependencies issues such as ruby versions or php/python versions used by these projects, but I think that can be addressed by installing different language versions and libraries in parallel in the same distro.

The stable release of distributions should separate packages into a stable repo and volatile ones carefully at the top layer, then you add those sub-repos underneath of each repo(security repo, third-party repo, etc).

Ubuntu, ownCloud, and a hidden dark side of Linux software repositories (PC World)

Posted Nov 10, 2014 23:31 UTC (Mon) by eean (subscriber, #50420) [Link]

Or just don't package it at all and let upstream do it. Like they are already.