|
|
Subscribe / Log in / New account

Supporting CentOS

By Jonathan Corbet
March 28, 2011
There are rumors suggesting that the CentOS 5.6 release is imminent - though that is something we have heard before. This release will certainly be welcome to numerous CentOS users, but there can be no doubt that its tardiness - and, in particular, the absence of CentOS 5 security updates caused by its delay - has been a bit of a wakeup call for those users. If this much-used distribution is to remain viable into the future, some important changes will need to be made and those who depend on it will have to step up their support.

There will be no shortage of CentOS users who would like to get their hands on the improvements and added hardware support to be found in the RHEL 5.6 and 6.0 releases. But the real problem is not delayed gratification; it is that there have been no CentOS 5 security updates since January 6, and only one since December 14, 2010. During this time, RHEL 5, on which CentOS 5 is based, has seen updates for dbus, exim, firefox (twice), gcc, hplip, java-openjdk, kernel (thrice), krb5, libtiff, libuser, mailman, openldap, pango, php, postgresql, python, samba, subversion (twice), tomcat5, vsftpd, and wireshark (twice). Since these updates are based on the 5.6 release, CentOS cannot easily pass them on to its users until they, too, have a 5.6 base. Since that base has been slow in coming, all those security updates have been blocked.

Some of these vulnerabilities are more severe than others, but there can be no contesting the fact that every CentOS 5 system out there is currently running with a significant set of known holes. That is not the sort of solidity and support that CentOS users will have been hoping for. Many of those users will, by now, be wondering whether CentOS is the right distribution to base their systems on.

The CentOS mailing list has been filled with users asking when updates would start flowing and why things have bogged down for so long. Some say that there are too many RHEL repackaging projects out there, and that CentOS should join forces with a distribution like Scientific Linux. Others blame the 6.0 release for distracting the project from its 5.x-based users - causing security updates for installed systems to languish in favor of a shiny new distribution that nobody is running yet. Still others complain that the project is insular, secretive, and hostile to new contributors. All of these claims may or may not be true, but they are not the subject of this article: there is another aspect to the problem that is unambiguous and clear.

Many people benefit from the work of the CentOS project, but at the top of the list must be managed hosting providers. Those companies get, for free, a solid platform which they can install on thousands of servers and sell to their customers. A site called tophosts.com maintains a list of the top 25 hosting companies; a look at that list leads to some interesting conclusions. Of those 25 companies:

  • One is a Windows-only provider.

  • Two offer "Linux" with no way, short of actually renting a server, of determining what flavor of Linux is involved.

  • Three appear to offer Red Hat Enterprise Linux only.

  • All of the rest (19 providers) offer CentOS.

(As an aside, it is amazing how hard many of these companies make it to find out what it is that they are offering to sell. Hosting provider web sites seem to all be designed by the same person; they are twisty mazes of little JavaScript functions, all alike.)

Represented on this list are the largest hosting providers in existence - though it must be said that the list is US-centric. Together, they account for many hundreds of thousands of systems, a significant percentage of which are running CentOS. That's a lot of business - a lot of revenue - which is being generated by CentOS-based systems.

The failure of CentOS - or even a significant tarnishing of its reputation - would reduce the value of the services offered by these providers. Other free Linux distributions exist, and some are entirely suitable for stable deployment situations, but many customers want a distribution which is compatible with RHEL. So said providers have a significant stake in keeping the perceived value of CentOS high. Perhaps it is time that some of them put some resources into supporting that value.

Said resources could certainly take the form of monetary donations to the project. But far better would be for these companies to hire somebody to work directly with CentOS and make it better. In return, they would reap all of the benefits that come with open source participation: they would have a better distribution to offer to their customers, they would get more influence over the direction of the project, their participation would enhance their reputation, and, crucially, they would improve their in-house expertise which could then be used to support their customers. All of the motivations for supporting free software development in other parts of the economy apply just as strongly to hosting providers.

A look at the CentOS Sponsors Page shows that quite a few hosting companies - including a handful of the big ones from the list described above - are supporting the project. In many cases, it seems, that support takes the form of a donated server. CentOS certainly needs servers and bandwidth, but those, alone, will not keep the distribution strong. Even the strongest contributor gets a bargain from Linux - nobody puts in as much as they get out. But one suspects that the hosting industry is getting a better deal than many. Now would be a good time for the top beneficiaries of the CentOS project to roll up their sleeves and put some serious time into making it better.


to post comments

silly question

Posted Mar 28, 2011 23:03 UTC (Mon) by coriordan (guest, #7544) [Link] (7 responses)

My silly question: Why is does a release like CentOS 6 take a lot of work?

I thought they just search-and-replace'd the logos and trademarks from Red Hat's enterprise GNU/Linux distro.

For security updates, I can see how that might be trickier. Do they have to get a Red Hat customer to pass them the updates? Does that customer risk losing their Red Hat service contract?

silly question

Posted Mar 28, 2011 23:07 UTC (Mon) by skvidal (guest, #3094) [Link] (6 responses)

the srpms for updates/security errata are all up on ftp.redhat.com, just like everything else.

ftp://ftp.redhat.com/redhat/linux/enterprise/6Server/en/o...

-sv

silly question

Posted Mar 28, 2011 23:14 UTC (Mon) by coriordan (guest, #7544) [Link] (5 responses)

Aha. Thanks.

So my question is then: why does any CentOS release take a lot of work?

silly question

Posted Mar 29, 2011 0:18 UTC (Tue) by cowsandmilk (guest, #55475) [Link] (3 responses)

my feeling is that a lot of it is QA.

See http://lists.centos.org/pipermail/centos/2011-March/10838...

You download the upstream src rpms, build them, but by default they link to different versions of libraries than upstream. While this doesn't necessarily mean it will be buggy or behave differently than upstream, it might. Now you have to go in and fix that. And you spend hours doing it for a build. Then rebuild and repeat.

As I understand it, if they release with it wrong, it can be next to impossible for them to release a fix and have yum upgrade to the fix. They have to have package numbering exactly like upstream due to third party packages that rely on those numbers, so they can't bump a number to get yum to upgrade a package. So, they want to make sure it is perfect before releasing it.

I have no inside knowledge of the project, so I won't speak beyond that.

silly question

Posted Mar 29, 2011 7:15 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

They can bump a version but they have do it differently inorder to not screw up the versioning. So if they want to update from foo 0.1-1 to a newer version, instead of doing a foo 0.1-2, they do a foo 0.1-1.el5-1 and if upstream does a foo 0.1-2, it will be considered a upgrade.

silly question

Posted Mar 29, 2011 13:16 UTC (Tue) by cowsandmilk (guest, #55475) [Link] (1 responses)

yes, they can do that, if necessary, but it would cause breakage with third party software.

see http://lists.centos.org/pipermail/centos-devel/2011-March...

The problem isn't with yum or the ability to create intermediate versions technically, the problem is third party commercial vendors match the string exactly, and when centos does foo 0.1-1.el5-1 , it will break the installers and/or startup scripts from those vendors. And the point of running centos is to not have those problems.

silly question

Posted Mar 29, 2011 13:28 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]

Derivatives already do tagging differently for various packages and there are ISV apps that string match /etc/redhat-release and it fails on them etc. It is a manageable problem if one wants to take the extra effort required.

silly question

Posted Mar 30, 2011 12:18 UTC (Wed) by Nelson (subscriber, #21712) [Link]

From a couple years lurking on their mailing lists and an interview on the FLOSS podcast, I think there are only a couple guys doing the actual work and they've got normal type jobs. Then there are some community type people that manage the website and speak for the project and such.

With something like Centos being as popular as it is, you'd sort of expect maybe 2 full time devs on it and a community of maybe 15-20 regular contributors. It's not clear that it's that way though. Then 5.6 vs. 6.0 become big decisions.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 28, 2011 23:11 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (21 responses)

To lighten up the mood, here's a flaimbait thread.

Why not just pour efforts into Debian instead? The only thing missing there are backports of new drivers into the stable branch.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 0:06 UTC (Tue) by drag (guest, #31333) [Link]

Because it would defeat the purpose of having a Redhat clone if you choose to copy Debian.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 0:29 UTC (Tue) by airlied (subscriber, #9104) [Link] (14 responses)

and decent QA.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 0:46 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (13 responses)

To be fair, I've encountered about 10 times more problems with CentOS/RHEL packages than with Debian.

The core packages in RHEL are solid. But anything other than core is questionable.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 5:48 UTC (Tue) by Darkmere (subscriber, #53695) [Link] (12 responses)

Since this is the flamebait thread:

Stable and solid like openssl? Well reviewed and understood packaging changes that get responded to in short time?

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 13:03 UTC (Tue) by ballombe (subscriber, #9523) [Link]

Look, this was the only way Debian had to find out how many high-profile websites were using it. Now, we know.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 13:24 UTC (Tue) by dskoll (subscriber, #1630) [Link] (6 responses)

OK, openssl was a huge fiasco. But the big advantage of Debian over CentOS is that the Debian developers are in charge of their own destiny, whereas CentOS (by design) is almost completely driven by decisions made by Red Hat.

Debian had problems in the past (and might have problems again in the future), but it has a long and distinguished track record and (importantly) is not at the mercy of a third-party corporation.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 14:40 UTC (Tue) by NAR (subscriber, #1313) [Link] (5 responses)

But the big advantage of Debian over CentOS is that the Debian developers are in charge of their own destiny, whereas CentOS (by design) is almost completely driven by decisions made by Red Hat.

If somebody wants a RedHat-compatible OS in order to run 3rd party binary-only software, this is not an advantage, but a disadvantage...

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 14:49 UTC (Tue) by dskoll (subscriber, #1630) [Link] (4 responses)

If somebody wants a RedHat-compatible OS in order to run 3rd party binary-only software, this is not an advantage, but a disadvantage...

If someone is crazy or unlucky enough to have to run 3rd-party binary-only software, that person should shell out for Red Hat Enterprise Linux. I don't believe most proprietary ISVs will give you the time of day if you're not running on an officially-approved platform.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 15:20 UTC (Tue) by kmccarty (subscriber, #12085) [Link] (1 responses)

If someone is crazy or unlucky enough to have to run 3rd-party binary-only software, that person should shell out for Red Hat Enterprise Linux.

Since based on my experience, it [RHEL] is probably cheaper than the 3rd-party binary-only software anyway...

no good deed goes unpunished?

Posted Mar 29, 2011 16:28 UTC (Tue) by dmarti (subscriber, #11625) [Link]

Yes, you want to help software vendors develop better habits so you end up with a more constructive industry. Reinforcing problem behaviors of the other vendor while punishing RHT for constructive behavior is not the way to get that.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 30, 2011 9:51 UTC (Wed) by theno23 (guest, #8859) [Link] (1 responses)

Just FYI, my company ships proprietary software as RPMs, that we support on CentOS and RHEL. The software is quite expensive, but most people still run it on CentOS - it needs quite a lot of hardware to run on, so running on RHEL is a signficant cost.

So I don't like like a complete capitalist pig, we also provide a lot of GPLv3 software.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 31, 2011 10:49 UTC (Thu) by jpnp (guest, #63341) [Link]

Out of interest, is your software only supported on RHEL/CentOS or do you also package for/support Debian or any of its derivatives?

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 15:12 UTC (Tue) by jubal (subscriber, #67202) [Link] (2 responses)

@darkmere – the only difference between Debian and $some_big_corpo is that you actually did learn about the issue and you did learn about it from (somewhat embarassed) Debian people themselves.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 15:41 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link] (1 responses)

As far as Linux distribution vendors are concerned, they disclose when a security problem happens and if there is a errata to be published, they will and not just hide it. Enough history here including

http://www.redhat.com/security/data/openssh-blacklist.html

There is no reason to suggest otherwise.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 23:39 UTC (Tue) by jubal (subscriber, #67202) [Link]

As far as Linux distribution vendors are concerned, they disclose when a security problem happens and if there is a errata to be published, they will and not just hide it.

Yes, and Red Hat seems to be much more diligent than your usual $big_corpo. (I did not want to make an impression that I'm writing about RH; the way Red Hat gives back to the community, and the quality of code produced there seem to be much above the average).

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 18:30 UTC (Tue) by flewellyn (subscriber, #5047) [Link]

Y'know, I can't really find fault with Debian in that one. Sure, it turned out to be a mistake, but a mistake that could have been easily prevented if the OpenSSL developers had

1) Left a comment in the code saying "We are deliberately using uninitialized memory here!",

or

2) Responded to queries on the mailing list about why that code was there.

Absent any kind of information to signal that this seeming bug was not, in fact, a bug, they did the reasonable thing in "fixing" it. Turned out it was a mistake, but how were they to know?

I know the OpenSSL folks insisted that was the wrong mailing list (why the correct one was not published is beyond me), but regardless, they communicated very badly. Rule number 1 in doing anything arcane in programming is, if you need to use an idiom that is usually bad practice or a source of bugs, COMMENT IT!

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 7:20 UTC (Tue) by ptman (subscriber, #57271) [Link] (1 responses)

There are newer kernels in backports.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 9:23 UTC (Tue) by tzafrir (subscriber, #11501) [Link]

And there are some newer drivers in point releases.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 16:57 UTC (Tue) by jone (guest, #62596) [Link] (1 responses)

ah Debian .. didn't they get divorced after ian took his job with Sun?

(you did mark this *FLAMEBAIT* ..no?)

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 29, 2011 19:29 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

Yeah. It's now Pkgian.

*FLAMEBAIT* CentOS - the new Debian!

Posted Mar 31, 2011 9:17 UTC (Thu) by jschrod (subscriber, #1646) [Link]

Because I use CentOS to check that our (open-source) software runs on RHEL without problems? I wouldn't be able to do that with Debian.

Supporting CentOS

Posted Mar 28, 2011 23:16 UTC (Mon) by smoogen (subscriber, #97) [Link] (26 responses)

My guess is that it is something of a "Tragedy of the Commons" (if I am using that phrase correctly). The hosting companies save money by using CentOS over a commercial vendor (SuSE, Red Hat, Canonical), and the amount of money it would cost to pay for a developer would basically get rid of any savings they are seeing. [Or in some cases, those 'savings' are what keeps the companies lights on so they could not hire anyone if they liked.]

So it feeds on itself as the resources are drained but little is put back. However, I don't know if any of this has anything to do with the CentOS delays or if all of a sudden several developers were 'hired' to work on CentOS they could in any quick time frame unless they were already familiar with the systems in place.

In the end, I hope things improve soon.

Supporting CentOS

Posted Mar 28, 2011 23:56 UTC (Mon) by dlang (guest, #313) [Link] (12 responses)

at the prices of RHEL it doesn't take that many servers before you start to get to the salary of one person, any hosting provider that is running a thousand servers can better afford to pay one person to work on the distro than they could afford to switch to RHEL. All of these hosting providers are running _far_ more than a mere thousand servers.

Supporting CentOS

Posted Mar 29, 2011 1:21 UTC (Tue) by zlynx (guest, #2285) [Link] (4 responses)

I haven't examined Red Hat's licensing deals very closely but I wouldn't be surprised if they could work out some kind of bulk license deal for a web hosting or virtual cloud server company.

It would seem insane to charge the full yearly support price for each virtual instance.

RHEL support levels?

Posted Mar 29, 2011 2:03 UTC (Tue) by dowdle (subscriber, #659) [Link] (3 responses)

Red Hat has few different products and subscriptions. See:

http://www.redhat.com/rhel/server/compare/

Basically they have "Server" where it provides for support for 4 VMs... and "Advanced Platform" where it provides support for unlimited VMs. If hosting providers are running mostly VMs, then I'm sure they'd be more interested in the unlimited option... where the number of VMs doesn't change the price... so no sort of licensing deal really needs to be worked out.

RHEL support levels?

Posted Mar 29, 2011 3:52 UTC (Tue) by dlang (guest, #313) [Link] (2 responses)

you still have to pay the license fee for each physical server.

even if they drop the price per server by 90%, the price per server would be >$100, at 1000 servers that easily pays for a developer. And I doubt that they get that steep a discount, or have that few physical servers.

RHEL support levels?

Posted Mar 29, 2011 6:09 UTC (Tue) by drag (guest, #31333) [Link] (1 responses)

It kinda sucks.

It would be nice if Redhat lowered it's pricing. But it would then undercut their enterprise sales. It all really depends on what will provide them the most profit. I wouldn't hazard a guess on what is best for them.

I've always thought that CentOS benefited Redhat because not everybody can afford to run Redhat supported versions on all their servers.

Say you ran into a situation were you ran a good sized IT department. Say in that IT department you had 'critical' systems that had special regulatory and/or tight turn around times. That is if they had a problem business requirements have you with 24/7 support and 2 hour turn around requirement for a fix. But you had a whole mess of systems that were not nearly so critical.

With the combination of CentOS + Redhat that can provide a cost effective solution were you maintain a more-or-less common platform across all your systems. You had your ass covered in case you got over your head on the critical systems, but did not have to blow your entire budget on all your systems.

If you had to do a mixture of Debian + Redhat systems, for example, then your not just dealing with licensing costs you have the additional overhead of now being forced to support multiple platforms. This raises your documentation costs, training costs, support costs, hardware aquisition costs... everything your IT department does all of a sudden gets more expensive.

If you cannot afford to go all-Redhat then your quite possibly going to be forced to abandon Redhat altogether and go with a third party support channel for Debian.

And, believe me, the costs associated with dealing with multiple platforms often pales in comparison to the cost of dealing with a specific platform's idiosyncrasies uniformly across all your systems. All systems have their issues.

I think that there are a significant number of organizations that run into situations like this. Where the option using CentOS potentially offers a very significant value added to Redhat licensing.

RHEL support levels?

Posted Mar 29, 2011 19:48 UTC (Tue) by dlang (guest, #313) [Link]

RedHat has a right to price RHEL however they want, I'm not arguing anything there.

I'm just pointing out that if you have lots of computers (like a large hosting company would), you can very quickly get to the point where what you are paying in license fees dwarfs the cost of hiring people.

RedHat works to maximize their profits by pricing it as high as they can, without making it so high that people decide to hire experts to participate in the community directly instead.

Supporting CentOS

Posted Mar 29, 2011 10:17 UTC (Tue) by NAR (subscriber, #1313) [Link] (5 responses)

On the other hand the CentOS-using hosting companies are competing with each other - maybe the money spent on an an extra engineer can be better spent on marketing (i.e. would return more sales)...

Supporting CentOS

Posted Mar 30, 2011 7:38 UTC (Wed) by Cato (guest, #7643) [Link] (4 responses)

Perhaps the solution is for hosting companies to donate money to a fund that is specifically to pay for an independent CentOS developer or tester. That way, no single hosting company has to cover the complete cost of the developer, and more of them can use a "we support CentOS" logo in their marketing, and most importantly CentOS gets more timely releases.

One of the key applications driving CentOS use in hosting is cPanel, which is by far the most popular control panel and doesn't support Debian/Ubuntu. Plesk and DirectAdmin, which seem the next most popular, do support Debian/Ubuntu and others.

Supporting CentOS

Posted Mar 30, 2011 8:27 UTC (Wed) by paulj (subscriber, #341) [Link] (3 responses)

Yes, and perhaps they could incorporate a body to manage that fund, maybe invest in R&D for CentOS. Hell, it could even turn a profit and make some money for its employees and investors.

Why has no one tried this already?

Supporting CentOS

Posted Mar 30, 2011 16:32 UTC (Wed) by smoogen (subscriber, #97) [Link] (1 responses)

Because it is not as easy as it sounds due to a standard chicken/egg problem.

1) You have to create a vehicle that businesses could 'donate' to. That has all kinds of legal/tax issues involved. Is it a service they are subscribing to (operating expenses), is it a product they are buying (capital expenses), is it a donation that they can write off, or does it become an extra expense that they can't.

2) You have to have a product they can see (ok in this case it is developers they are buying into).. Have they done anything yet to show its not a flim-flam job

3) You have to have a sales force that they can trust. Because you are going to have to sell this crazy idea to people who think that the only way they are "winning/surviving" is by not paying for something in the first place.

Supporting CentOS

Posted Mar 30, 2011 20:43 UTC (Wed) by jrn (subscriber, #64214) [Link]

I think paulj was suggesting that it has already been done by a certain Red Hat, Inc. :)

Supporting CentOS

Posted Mar 31, 2011 10:43 UTC (Thu) by Cato (guest, #7643) [Link]

I'm just talking about a legal entity to receive the funds, or even a shared bank account with multiple signatories, and a Paypal account. There's no need to make this more complex than is needed.

Supporting CentOS

Posted Mar 29, 2011 12:28 UTC (Tue) by Escubar (guest, #39034) [Link]

RHEL is indeed quite expensive, but you should check out SUSE Linux Enterprise Server. Easy to handle virtualization policy and yes there are volume discounts which make it affordable to hosters and others.

Supporting CentOS

Posted Mar 29, 2011 0:16 UTC (Tue) by Lennie (subscriber, #49641) [Link] (8 responses)

I work at a hosting company, we run Debian. It does not have the problems the CentOS folks seem to be having.

Being without security updates would scare the hell out of me.

Supporting CentOS

Posted Mar 29, 2011 8:45 UTC (Tue) by seyman (subscriber, #1172) [Link] (7 responses)

Note that Debian has had the same problem in the past.

Supporting CentOS

Posted Mar 30, 2011 10:08 UTC (Wed) by Cato (guest, #7643) [Link] (4 responses)

That was 6 years ago, so I hope Debian's security update process is better sorted out now.

Ubuntu server also looks quite good given its 5 year support lifetime and generally quick updates, plus the option of paying for support on a few critical servers - sort of like CentOS+RHEL combined as there are no software differences between the free and paid-support versions. If only there was cPanel support for typical users - still at least a year away in the cPanel roadmap: http://forums.cpanel.net/f145/debian-support-ubuntu-serve...

Supporting CentOS

Posted Mar 31, 2011 14:10 UTC (Thu) by seyman (subscriber, #1172) [Link]

<p><em>That was 6 years ago, so I hope Debian's security update process is better sorted out now.</em></p>

<p>I don't know. Since then, we've had the openssl fiasco and <a href=http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=611176"">the reaction</a> to <a href="http://lpsolit.wordpress.com/2011/03/04/debian-takes-secu...">Bugzilla security fixes</a> leaves me thinking that Debian still has problems with security updates.</p>

Supporting CentOS

Posted Mar 31, 2011 14:11 UTC (Thu) by seyman (subscriber, #1172) [Link] (2 responses)

That was 6 years ago, so I hope Debian's security update process is better sorted out now.

I don't know. Since then, we've had the openssl fiasco and the reaction to Bugzilla security fixes leaves me thinking that Debian still has problems with security updates.

Supporting CentOS

Posted Apr 1, 2011 0:14 UTC (Fri) by Lennie (subscriber, #49641) [Link] (1 responses)

I think the OpenSSL fiasco wasn't just Debian, I think the OpenSSL had a part in that too.

Supporting CentOS

Posted Apr 1, 2011 8:05 UTC (Fri) by seyman (subscriber, #1172) [Link]

The problem isn't so much who's to blame but the fact that Debian does, on occasion, have the problems that Centos is having right now.

Supporting CentOS

Posted Apr 1, 2011 12:28 UTC (Fri) by zack (subscriber, #7062) [Link] (1 responses)

> Note that Debian has had the same problem in the past (https://lwn.net/Articles/144446/).

You know, Debian is an almost 18-year old distro and the episode you're pointing at is 6 years old. By looking at Debian's past you can find *many* mistakes, statistically many more than those you could find in the history of a younger distribution.

From Debian's mistakes, many distros out there have learned how do better. Including Debian.

You can find the current track record of Debian security updates at http://www.debian.org/security/ (which, just as an example, shows an average of more than 1 update/day for March 2011).

-- Stefano Zacchiroli

Supporting CentOS

Posted Apr 1, 2011 12:37 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

That's not a very useful metric. It shows Debian is actively releasing updates that fix security issues but doesn't help show that issues are being handled in a timely fashion. If a distro wants to publish metrics for security errata, look at

http://www.awe.com/mark/blog/20110117.html

Supporting CentOS

Posted Mar 29, 2011 2:54 UTC (Tue) by robbiew (guest, #57380) [Link] (3 responses)

Small correction, Canonical does not charge licensing fees for Ubuntu Server, so the financial cost of running Ubuntu Server is equal to that of running CentOS....$0.

Supporting CentOS

Posted Mar 29, 2011 5:25 UTC (Tue) by SEJeff (guest, #51588) [Link] (2 responses)

Ubuntu is free... Sometimes, you get what you pay for.

Supporting CentOS

Posted Mar 29, 2011 7:44 UTC (Tue) by rvfh (guest, #31018) [Link] (1 responses)

Same for CentOS. If you want the latest RedHat release and the security updates in a timely manner, than what you want it RedHat.

Supporting CentOS

Posted Mar 31, 2011 10:45 UTC (Thu) by Cato (guest, #7643) [Link]

The difference with Ubuntu is that you can get updates at the same time as 'enterprise support' customers, whereas RHEL updates are filtered through the CentOS process.

Supporting CentOS

Posted Mar 29, 2011 2:27 UTC (Tue) by dowdle (subscriber, #659) [Link] (2 responses)

Maybe they are working more slowly on purpose to put their users in a bind in an effort to get more support from the community... or maybe not.

Supporting CentOS

Posted Mar 30, 2011 17:30 UTC (Wed) by orev (guest, #50902) [Link] (1 responses)

Users are constantly asking how they can help, and they are usually treated with disdain and contempt. That was one of the issues this article decided to skip.

Supporting CentOS

Posted Apr 8, 2011 7:59 UTC (Fri) by spongy (guest, #59953) [Link]

"...disdain and contempt.."

Ahh, that must be the arrogant Johnny Hughes. I see he is still there.

Supporting CentOS

Posted Mar 29, 2011 2:51 UTC (Tue) by robbiew (guest, #57380) [Link]

Given how much AWS now relies on CentOS for their Linux images, it would seem in their best interest to help.

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 29, 2011 10:14 UTC (Tue) by jubal (subscriber, #67202) [Link] (7 responses)

If you follow the CentOS development mailing lists, you'll see the pattern of regular flamewars regarding the delays in shipping various versions and lacking security upgrades; the reasons cited for that are usually manpower and needing more time for QA.

Yet when you look at CentOS website, you don't find too much about how to contribute work to the distribution core; on the front page there's virtually no information on how to participate in QA. Hell, even on the wiki you have to explicitely hunt for QA, because there's no link from the front page!

And the wiki contributions page doesn't mention development/preparation of the CentOS core at all.

To me it seems, that CentOS dev team looks like a pretty closed gentleman's club with no desire to get more people aboard. And that is unfortunate.

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 29, 2011 11:44 UTC (Tue) by sgros (guest, #36440) [Link] (1 responses)

I always wondered why no one forked CentOS? Or starts similar builds based on Scientific Linux? And just a note, under "no one" I mean group, not single person...

Also, it wasn't so long ago when CentOS had problems with main developer. And back then things looked bright and promising. What happened in the mean time? Maybe grumpy editor should investigate. I would trust him to offer independent and non-biased view in this whole mess. And also, people will respond to his mails, unlike to the mails of majority of others...

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 30, 2011 14:04 UTC (Wed) by Nelson (subscriber, #21712) [Link]

One thing that is different between Centos and Scientific Linux is that Centos' goals are to make a 100% compatible RHEL clone. Once a build machine is able to fully build it, building the packages isn't too terribly difficult. I've been left under the impression that the technical work is more of a one time sort of effort and once it's done, there isn't much to help with. But yeah, they could do some better community management. Seems like someone should be able to help them spin up a foundation or something and at least make a platform which could be used to pay those silent gentlemen that are doing the work.

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 29, 2011 15:58 UTC (Tue) by butlerm (subscriber, #13312) [Link]

To me it seems, that CentOS dev team looks like a pretty closed gentleman's club with no desire to get more people aboard. And that is unfortunate.

That is certainly my impression. It is not at all obvious how to contribute at all. They are not even accepting financial contributions at the moment.

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 29, 2011 16:18 UTC (Tue) by danieldk (subscriber, #27876) [Link]

It is. I was part of the CentOS team for some time (2007-2008). Although some people within the team wanted to transform CentOS into an open non-profit project analogous to Debian, it was not really possible to get this off the ground.

If CentOS wants to be viable in the long term, it needs to consider such changes. If you continue as a group of three people who roll out all the core packages, there is always the possibility that a major version is frozen for some time if that person cannot commit the necessary time (or god forbid, gets run over by a bus).

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 29, 2011 19:59 UTC (Tue) by ESRI (guest, #52806) [Link]

Where I think CentOS could really use some volunteers is in the community management area.

Most of the volunteers are extremely technical, and very good at what they do, but the whole "how to contribute" thing -- identifying needs, organizing teams, etc would probably better be performed by someone not super heavily involved in the technical heavy lifting.

I'm _sure_ there are places where more bodies -- even if they're not as technically proficient, could help a lot. Just need someone to get it all figured out.

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 29, 2011 22:03 UTC (Tue) by mmarlowe (guest, #11374) [Link]

Yeah, I run a small consulting company which has a hosting business on the side, and we historically depended on RedHat/RHEL. However, when virtualization reared its head, we ended up spending most of our discretionary licensing/commercial support budget on VMware and SAN Storage. Then, the concept of virtual appliances, and dedicating virtual machines for individual services came along and it became clear that we might see a major expansion in the number of systems we managed (even if the number of physical boxes stayed relatively small). The best arrangement we could find with RedHat was roughly $1K/socket/yr for RHEL licensing and that just wasn't scalable.

We looked into CentOS but the state of its website, the random delays in releasing security patches, and the fact that we might incur unknown compatibility/QA issues made us decide that we would be much better off spending people time customizing and supporting OS like gentoo into tailor made servers than sticking with redhat based os's. The non-redhat systems may take a little more work, but the transparancy, open communication, and ability to become a part of the development teams help make up for it. Plus, there are no more major upgrades like RHEL5->RHEL6 - instead we define which software versions to lock down and control our own release schedule for everything else.

On a side note, thank you LWN.net for providing daily updates for security errata :)

centos' inherent problems: (lack of) transparency and nih syndrome

Posted Mar 30, 2011 1:17 UTC (Wed) by nikarul (subscriber, #4462) [Link]

I started investigating a little when it became clear that 5.6 updates weren't going to be forthcoming. What I found wasn't encouraging.

Like you, I didn't find any obvious way to contribute. Worse, I got the impression that they didn't *want* any help. The fact that pleas for timely security updates were met with phrases like "self entitled users" and "if its that critical get RHEL" didn't help either.

Well, it is that critical for me. And I will be moving to RHEL, or other distros as I see fit. But I wish I could contribute instead of migrating to another distro. It doesn't seem like there's a bright future for CentOS as a serious server distro at this point. I can only hope that changes, and soon.

Supporting CentOS

Posted Mar 31, 2011 13:12 UTC (Thu) by asherringham (guest, #33251) [Link]

A few years ago, my company needed to get a web server up and running quickly, with security being of primary importance. The (Linux) options available included RHEL, Centos and Ubuntu. Because I knew about the care and attention Redhat give to RHEL, and its security track record, I chose RHEL.

But when it came to testing a configuration and preparing for setup and deployment, I used Centos (in a VM). It's likely that if Centos was not available, I would not have chosen RHEL (I know Ubuntu better for instance). Centos gives you an easy way to jump/deploy to RHEL.

An important thing the Centos project should look at doing is organising themselves such that they can accept financial help e.g. non-profit company etc. They're not taking monetary donations just now. There might be other free software organisations that could help out here e.g. Software in the Public Interest (SPI).

Supporting CentOS

Posted Mar 31, 2011 20:12 UTC (Thu) by gswoods (subscriber, #37) [Link] (2 responses)

What I want to know is, why hasn't Scientific Linux encountered the same problems? Have they got paid developers, or do they just do a better job of "community management"?

Supporting CentOS

Posted Mar 31, 2011 20:23 UTC (Thu) by asherringham (guest, #33251) [Link]

From my understanding, Scientific Linux is developed by and for (primarily) an academic audience i.e. CERN, university departments. I think people working in these organisations often have more time to spend on creating, configuring and tailoring "tools" - like a Limux distribution. More people, more leeway. Probably as simple as that.

Supporting CentOS

Posted Apr 2, 2011 0:56 UTC (Sat) by LightDot (guest, #73140) [Link]

Scientific Linux has four full time, paid developers, AFAIK.

Why CentOS has allowed itself to become dependable on basically "one and a half" part time developer is beyond me. Considering what kind of user and resource potential it has.

Don't get me wrong, CentOS developers are great, as far as quality of the work goes. And their contribution is much appreciated.

But the project itself is run badly. It's understaffed and communication between the developers and the community at large is really bad. At times even abusive. Very knowledgeable, experienced and mature forum moderators (most of them also a part of the QA team) are the only thing that keeps the project from imploding. But even themselves are often kept in the dark or prevented from disclosing all information to the larger community.

That's my impression of the current state of things.

This problems seem to be present for a while now and they do get more attention in time of new releases, hiding beneath the "business as usual" feeling afterwards. But the problems have been worse and worse, release delays greater and greater.

So unless something changes before upstream reaches 7, CentOS is not going to fare well.

Supporting CentOS

Posted Apr 3, 2011 2:11 UTC (Sun) by wookey (guest, #5501) [Link] (1 responses)

I'm quite surprised that nearly all the major hosting providers run RHEL/Centos, but not Debian. Is there really no demand for Debian-based hosting? I certainly wouldn't use anything else, and my provider provides it and always has done (they doalso provide RHEL/RHEL-alike).

I manage/help manage about 15 Debian boxes of one sort or another, and it's very civilised. There is one place where I have to deal with a RH box and it's a major pain the bum (this is almost entirely due to familiarity, rather than there being anything wrong with RH setups, particualrly). I'm just surprised that 'everyone' (approximately) gets to use RH or RH for their hosting.

Supporting CentOS

Posted Apr 5, 2011 2:58 UTC (Tue) by VelvetElvis (guest, #69142) [Link]

I use Debian as well, but I could see where its life cycle could be too short for a lot of users. Simply migrating a wikifarm using the Debian MoinMoin packages every three years can be a pretty big hassle.

I'd much rather they have a pre-established security support timeframe rather than the unpredictable. "one year after the next release." You can't plan around that.

Supporting CentOS

Posted Apr 7, 2011 18:43 UTC (Thu) by herrold (guest, #54970) [Link] (2 responses)

From the article:
there can be no contesting the fact that every CentOS 5 system out there is currently running with a significant set of known holes

Why does LWN propigate FUD like this? A well-managed general CentOS unit exposed to the internet (or even just to unknown hostile 'trusted insiders' in a private network segment) is not running without iptables and wrappers, and is not exposing all services on all possible ports. There is no remote vulnerability in the kernel, in iptables, in wrappers, nor in the sshd as regularly configured

If the author of this statement (seemingly Jonathan Corbet) wishes to demonstrate such, please contact me, and I'll provide a unit at an IP for him to test until the cows come home. If he can compromise, it, I'll readily update this comment acknowledging the same. I trust he'll retract the quote above if (and I antcipate, when) it proves false

-- Russ herrold

Supporting CentOS

Posted Apr 7, 2011 19:15 UTC (Thu) by Trelane (subscriber, #56877) [Link]

there can be no contesting the fact that every CentOS 5 system out there is currently running with a significant set of known holes
A well-managed general CentOS unit exposed to the internet (or even just to unknown hostile 'trusted insiders' in a private network segment) is not running without iptables and wrappers, and is not exposing all services on all possible ports.

So what you're saying is that yes, there are holes, but they're covered with a piece of sheet metal.

You're not contesting the existence of holes; rather, you're claiming they're not exploitable holes due to other security measures in place.

Supporting CentOS

Posted Apr 7, 2011 20:32 UTC (Thu) by LightDot (guest, #73140) [Link]

How about CentOS servers providing shared hosting accounts? And those running virtual private servers, either KVM, openVZ or XEN?

Are you basically saying that servers with very very limited and firewalled network services, with no untrusted local users, aren't vulnerable? That's nice but nearly not good enough.


Copyright © 2011, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds