Weekly Edition
Return to the Distributions page
| |||||||||||||
Debian LTS?The allure of "long-term support" (LTS) releases for a distribution is strong. We have seen various distributions struggle with the idea of an LTS release over the years and, of course, Ubuntu has five years of official support for its LTS. More recently, Debian started discussing a "Debian LTS" on the debian-devel mailing list. As is often the case, the "many years of support" idea may well run aground due to a typical problem for community distributions: people power. The posting of a June announcement that the DreamHost hosting service would be moving from Debian to Ubuntu LTS sparked the discussion. The switch was made because the "release cycle for Debian is stable, but it's not long enough for us to focus on stability", according to DreamHost. The support window for a Debian release is one year after the next stable release, which generally works out to around three years. For some—DreamHost included—three years is not enough time. The main problem for extending the Debian support window, unsurprisingly, is security updates. Though, as Russ Allbery pointed out, some Ubuntu LTS users may not actually be getting the full support they are expecting, because it is only the packages in the "main" (and not "universe") repository that get that support.
Rather, they just blindly assume that LTS having security
support for five years means that, as long as they regularly upgrade, they
don't have to worry about security.
They therefore end up running various non-core software with open security vulnerabilities. But that problem is not limited to Ubuntu, as Paul Wise noted. Currently, Iceweasel (Debian's version of Firefox) is no longer supported for squeeze (Debian 6.0) and other packages have fallen out of support during the one-year oldstable support period in the past, he said. The sheer size of the Debian package repository makes the support problem that much harder, which is presumably why Ubuntu has focused its support on a smaller subset. There also seems to be some disagreement among Debian developers with regard to handling security updates. Some, like Pau Garcia i Quiles see security updates as the responsibility of the package maintainer (with some assistance from the security team). Others, like Steve Langasek, believe it is a security team function with input and assistance from the maintainer. In the end, it may not really matter; without volunteers to prolong the support, no LTS is possible. As Ian Jackson put it: "unless and until we have people who volunteer to do the security support for an LTS, we won't have an LTS". There is more to it than that, however, according to Allbery. He argued that beyond two years, security support is somewhat illusory, even for the enterprise distributions. Once the upstream projects have moved on—and by two years after a release, they mostly have—it becomes harder and harder to support older releases.
I'm painfully aware of the steep cliff
dropoff of upstream support for security fixes once you go beyond two
years after the release of the software. Beyond that point, even if you
do get security fixes, you're probably getting fixes that have been
backported by people with only a vague knowledge of the code, fixes that
often have not been thoroughly tested or tested by someone who uses the
software in question and that upstream has never looked at and will
disclaim any knowledge of or support for.
As painful as it is, if you are worried about security in a production environment, falling more than a couple of years behind current distribution releases is probably not in your best interests no matter what security support you supposedly have. Wookey took a different tack, however, noting that organizations like DreamHost should, if they are interested in longer-term support, be funding the work. Philip Hands agreed, pointing out that it is "not an exciting task that volunteers can be expected to flock to". He suggested that interested companies band together:
If that made the somewhat more enlightened
companies band together to share the LTS workload amongst themselves
somehow (possibly by having a limited distribution model of some sort,
restricted to members of the mutual-support-club) then that would be no
bad thing either.
Hands also cautioned that community efforts at long-term support could be detrimental to the distribution as a whole: "I suppose it would be nice if we could provide this long term support, but really it's going to consume scarce volunteer effort which will almost certainly have a negative impact on the progress of Debian proper." In many ways, that is the overall conclusion in the thread. While there were numerous maintainers volunteering to support their packages for longer, it really requires a distribution-wide commitment (or that of a dedicated team), which doesn't seem to be appearing. LTS releases are clearly popular with certain segments of the Linux community, but it is a big effort—one that is difficult for a distribution made up of volunteers to commit to. Even distributions that are supported by companies, like Fedora, find that commitment to be difficult, though Ubuntu LTS and openSUSE's Evergreen do provide counterexamples. It doesn't seem unreasonable that those who want longer support help make it happen, either via their efforts or their money. The latter is one way to look at the enterprise distribution model. For Debian, though, money alone won't solve the LTS problem—it will require a fair amount of effort from those who need it. (Log in to post comments)
Debian LTS? Posted Aug 29, 2013 12:38 UTC (Thu) by pabs (subscriber, #43278) [Link]
It is interesting to note that at least one company has now committed employee time to help out with Debian security as a result of this discussion. I also believe at least one existing Debian security team member is paid by his employer for doing Debian security work.
http://lists.debian.org/20130829095957.GB14146@ftbfs.de
If anyone reading the article is able to commit time to help out with Debian security, please check out my reply:
Debian LTS? Posted Aug 29, 2013 18:56 UTC (Thu) by vonbrand (subscriber, #4458) [Link] How many packages does Debian have? How many packagers? I'd suspect that the brainpower needed to look after a package's security updates competently is rougly comparable to the one needed to keep up with upstream. So a few people working on this aren't enough. Not by a very long shot. (Same applies to any distribution, obviously)
Debian LTS? Posted Aug 30, 2013 1:03 UTC (Fri) by drag (subscriber, #31333) [Link]
This is why distributions should not focus their attention on packaging and maintaining the software themselves, but instead focus their attention on the core operating system platform and work with upstream on packaging their software, upstream, for multiple distributions.
'Linux ecosystem' as a whole has more then enough manpower to support significant software going back many years if upstream is good to work with on this subject and the software is well enough designed.
Obviously some projects are going to be more troublesome then others to maintain, but that is just going to be something end users are going to be forced to take into consideration when they choose the software stacks that their 'enterprise' applications depend on. It is very simply not worth it for some upstream vendors to support older versions indefinitely... their software was not designed for it and often they lack the desire for it or maintaining all released versions for 5 years has zero benefit to anybody. Other projects are different and it is sane to support their major releases for a very very long time.
The way things currently stand were distributions retreat to their respective silos and try to build and support the entire available software ecosystem in perfect equality until X many years is complete lunacy.
Individual distributions lack the manpower, ability, knowledge, expertise, and just plain intimacy with each and every software package that they build and then drop off on their ftp server. Granted some packagers are the same people as upstream and some packagers participate heavily in various projects, but I can't imagine that would make up a significant amount of the software ditributions distribute. As it is now they can barely keep up with the software packages that they currently support for their latest and greatest and there is a huge amount of software that goes completely neglected by distributions.
It is never going to work the way things are and unless people are willing to change how they organize themselves then it's never going to get better.
upstreams don't have the manpower Posted Aug 30, 2013 4:24 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
upstream projects don't have the manpower to maintail 'LTS' versions for the 5+ years that some people want, most of them don't even really support the 3 years or so that the current Debian Stable claims to provide. This isn't just small projects, the Linux Kernel doesn'treally provide support for this long.
Debian LTS? Posted Aug 30, 2013 6:13 UTC (Fri) by anselm (subscriber, #2796) [Link] This is why distributions should not focus their attention on packaging and maintaining the software themselves, but instead focus their attention on the core operating system platform and work with upstream on packaging their software, upstream, for multiple distributions. Right. As if upstream would be interested in maintaining packages for multiple distributions for years and years. Many upstream projects barely have the manpower to move their software forward, and new development is a hundred times more alluring than maintaining old code.
Debian LTS? Posted Aug 30, 2013 17:48 UTC (Fri) by iq-0 (subscriber, #36655) [Link]
So true. Most projects don't support any older version. If there's a bug, than it's fixed in the next release. You are expected to run the latest if you expect quality from those projects.
The problem is that this model fails when you have lots of interdependencies. This is where distributions step in and try to get the correct mix that can be regarded as stable.
But in the process they have to maintain version stability (partly because people expect their things not to change (any change can change something for the enduser, often positive but the negative ones are not tollerated) and partly because it's almost impossible to oversee the consequences of the whole when something is changed).
Debian LTS? Posted Sep 7, 2013 17:19 UTC (Sat) by faheem (guest, #47574) [Link]
Both these links bring up the same message for me.
Debian LTS? Posted Sep 12, 2013 9:20 UTC (Thu) by pabs (subscriber, #43278) [Link]
Woops, my reply is here:
Debian LTS? Posted Sep 1, 2013 2:53 UTC (Sun) by cas (subscriber, #52554) [Link]
personally, i think that the whole "LTS is good for security" idea is seriously misguided.
IMO, the best way to maintain a secure system is to stay ahead of the bugs, ahead of the attackers - which means running (slightly behind) the leading edge of development.
in short: a moving target is harder to hit.
i've been running all of my desktop machines, and almost all of my production servers (thousands of them over the years - physical and virtual) on debian sid since the mid 1990s - with regular, routine upgrades every 2-3 months on average. before tools like puppet came along, i wrote my own shell and perl wrappers around ssh to automate mass upgrades of multiple servers - test the upgrade on test/dev machines first (which also gives first-hand experience of any required config changes so i can write a perl or sed script to automate the config update) and then automate the rollout to production if there were no major problems. in fact, i still use a mixture of my own scripts and puppet today.
the end result is that my servers tend to be at least 6+ months ahead of the script kiddies and other attackers, and I have the features, bug-fixes and improvements of the latest (or almost-latest) versions of all the packages I'm running. of course, i also have the exciting new bugs instead of the boring old ones...but only for a little while, if the bug is a big problem I either skip that update on production servers, or it gets fixed on the next routine upgrade. I don't have any *more* bugs, I just have *different* ones...and while i have no hard data to prove it, i suspect that I actually have *fewer* bugs this way.
for some rapidly-developing programs with bug-fixes and improvements appearing weekly or even daily, this is a huge benefit. qemu, kvm, libvirt and friends have been a great example of this - the pace of change and improvements with these packages really makes keeping up-to-date worthwhile.
I really hate to tempt fate by saying this, but i've never had one of my servers compromised - but i have had to fix (i.e. backup, erase, rebuild from scratch) servers run by people who think that "stable" or "LTS" is a good security model so end up running code which has had known exploits available for several years.
Another advantage is that instead of having to deal with a huge number of changes all at once every year or two, I deal with a much smaller number of changes all the time - this is much more manageable, and far less prone to causing downtime or problems. it's easier and much less work to fix one or two little things at a time than have to fix hundreds of things on a major update.
This is also made easier by the fact that dealing with upgrades is a routine experience that I get constant practice at, so I'm much better at it than if it was an unusual event, something I only had to do once every few years. familiarity breeds competence :-).
occasionally, when there are major changes in capabilities and especially configuration and compatibilty between one version of an important (to the particular server) package, I may have to Hold a package or set of packages for a while (a few weeks or months) so that an 'apt-get dist-upgrade' will leave them alone while the rest of the system is updated - e.g. apache/php/etc on a web server, samba or nfs on a file server, postfix/amavis/spamassassin/etc on a mail server, and so on.
Debian LTS? Posted Sep 1, 2013 3:30 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
I agree with this. I haven't run my servers where everything gets upgraded as frequently as cas, but I've done a similar upgrade cycle on the packages that provide the functionality that my servers are there to provide, including the kernel (and the other packages aren't exposed to the network, so they aren't as much of a security problem to leave on whatever version)
As cas says, this approach makes upgrades a routine thing that you get really good at rather than something that requires special planning.
another advantage is that when you do run into problems (usually on your test/non-production boxes, but occasionally on a production box for a bug that only shows up under heavy load), it's far easier to contact the upstream developers and work with them to get the bug fixed than to report it to a distro against a version that upstream no longer supports and wait for things to get fixed (I've had bugs sit for years this way)
like upgrades, if you do custom packages routinely, they become no big deal
so for the last several years, my servers have been Debian Stable + custom kerneol + custom packages for the things that I care about.
Debian LTS? Posted Sep 2, 2013 16:42 UTC (Mon) by intgr (subscriber, #39733) [Link]
> personally, i think that the whole "LTS is good for security" idea is seriously misguided.
Yes, but you are missing the point. The real reason is "LTS is good for stability" and "we need security as well" is an afterthought.
> i wrote my own shell and perl wrappers around ssh to automate mass upgrades of multiple servers - test the upgrade on test/dev machines first
As always, it depends on what you do. This may work well if you mostly only use distro-supplied components.
If you have tons of custom legacy applications that depend on distribution components and most applications have no meaningful way to automate testing, it becomes very hard to guarantee stability with updates. And you will almost always find out too late, in production, since it would take too much human time to thoroughly test every feature of every application on the testing setup every time. I hate to use this world, but I guess that's "enterprise".
Truth be told, most of these legacy applications are insecure anyway, but there's little incentive for attackers to target something that only has 1 installation in the world.
> of course, i also have the exciting new bugs instead of the boring old ones [...] I don't have any *more* bugs, I just have *different* ones
Old bugs are good bugs. People know them and know how to work around them. It's regressions that actually cost people time.
Debian LTS? Posted Sep 2, 2013 17:12 UTC (Mon) by mathstuf (subscriber, #69389) [Link]
> Truth be told, most of these legacy applications are insecure anyway, but there's little incentive for attackers to target something that only has 1 installation in the world.
Unless that one installation is hosted by a highly lucrative target.
|
|||||||||||||