LWN.net Logo

The Fedora-Red Hat Crisis (Datamation)

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 10, 2008 21:44 UTC (Wed) by BrucePerens (subscriber, #2510)
In reply to: The Fedora-Red Hat Crisis (Datamation) by smoogen
Parent article: The Fedora-Red Hat Crisis (Datamation)

Here's a direct quote from the Spafford paper you sent me:

The problem occurs when a flaw is discovered and the owners/operators attempt to maintain (indefinitely) the sanctity of the system by stopping disclosure of the flaw. That is not generally going to work for long, especially in the face of determined foes. The owners/operators should realize that there is no (indefinite) security in keeping the flaw secret.
The 2006 Debian penetration was to gluck.debian.org, a system hosted at Hewlett-Packard in Colorado. The SSL issue this year very definitely effected systems in the United States. Debian did not have to clam up. Debian is part of SPI, which is a U.S. 501(c)3 non-profit corporation.

If Red Hat is constrained by an investigation in progress, they should be able to at least say that much. Let's not try to hide them behind creative misinterpretation of the law.

Bruce


(Log in to post comments)

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 10, 2008 22:09 UTC (Wed) by smoogen (subscriber, #97) [Link]

Actually in the investigations I have dealt with, we were not able to even say we were working with law enforcement until they approved it. They instead told us we could say we had an issue and then later when they decided it would not interfere with their investigation we were able to say "we were working with law enforcement." [By the way, I have no knowledge if Red Hat contacted law enforcement or not.. ]

The difference I am trying to make between the SSL code and the Red Hat issue is that the SSL was a coding error with security implications, and the other was a criminal issue with security implications.

It would be better to compare how the 2006 break-in was dealt with (when was it reported to authorities, did the authorities believe the damage affect national security, stockholders, etc. When was the full-disclosure of the event made in comparison to the detection? etc). In that case RH/Fedora probably comes up wanting again.. but it is a better apples to apples.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 10, 2008 22:40 UTC (Wed) by rahvin (subscriber, #16953) [Link]

It would be HIGHLY unusual if the FBI wasn't involved. They would be risking serious class action shareholder lawsuits if they didn't call the FBI immediately after discovering the breach. RedHat likely has software in some top government agencies, including the NSA, DOD and CIA. Because of this the breach is probably being treated just like a breach at Microsoft would be treated. A full criminal investigation has probably already begun and everyone is gagged because as you said the FBI arrests and prosecutes people that reveal information before they have completed their investigation.

Frankly RedHat's behavior indicates to me that the FBI is involved. The press release reads just like some FBI agent scrubbed it of every single detail that wasn't directly relevant to notifying customers of issues that will affect them. In time we will know what happened, but we need to give them time for the FBI to relax the restrictions. I would say that if we don't have at least a mostly complete release of details in 6 months that it's probably time to start saying they aren't doing things the way they should. Otherwise like I said before I think the author and others are jumping the gun on putting RedHat's feet to the fire. And I think we should at least accept that RedHat has always had pretty good judgement and that if there was anything that directly affects customer security it has been released and the remainder is just details of how the break in occurred which don't necessarily have an impact on customer security.

And Bruce, there is a significant difference between a non-profit corporation who misconfigured software and a publicly traded company dealing with felony computer crimes. RedHat has numerous restrictions and obligations as a result of being publicly traded that Debian doesn't have to deal with. And those restrictions aren't just financial, they deal directly with what shareholders are told, how they are told and how criminal behavior is handled. Making a misstep could cost RedHat millions in SEC fines or shareholder lawsuits whereas there were no such implications to Debian.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 10, 2008 23:05 UTC (Wed) by BrucePerens (subscriber, #2510) [Link]

To compare apples to apples, you can see Debian's 2006 server compromise, which occurred to a system in the bastion (outside of the corporate firewall) at HP Fort Collins. They handled it much more openly than Red Hat.

The restrictions on publicly traded companies really act both ways. Destroying customer confidence by quashing discussion of a security issue will get you sued just as readily as saying too much about it.

Perhaps the bottom line here is that Fedora isn't as good a structure as Debian for maintaining the core of a Linux distribution, because Fedora is too close to the publicly-traded corporation to have freedom of action. Good engineering gets done when you have that freedom.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 11, 2008 0:04 UTC (Thu) by smoogen (subscriber, #97) [Link]

Yes, customer confidence is all a company selling/distributing Linux for the enterprise trades on. The questions I have from the Debian break-in that I could not see were:
1) Was the problem reported to police?
2) What was their take on it?
3) What would have HP done if the systems had been inside of a corporate firewall?

My biggest problem with your comments is that you are taking advantage of a person who can't speak for your public schadenfreude. I expected better of you versus cloning ESR. I expected you to write up a treatise of how a public group should handle security incidents.

Instead you seem intent on making non-Debian's regret defending Debian against a-holes who posted "oh what a bunch of dorks" when the SSL issue or the Debian 2006 breach occurred.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 11, 2008 0:17 UTC (Thu) by BrucePerens (subscriber, #2510) [Link]

I don't know what Debian's interaction with the police was.

Their take on it is public knowledge, just use google.

I have seen HP handle a security issue horribly, and did my best to correct the problem while I was at HP. I can't take responsibility for how they behave today, except to note that I was really glad I didn't work there any longer the day their general counsel took the 5th in front of Congress and on national TV.

Red Hat is eminently able to speak for itself. They choose not to. I have been making it clear that the Debian way was the prototype for a public group handling security incidents. And hold the ESR insults, please.

Debian took its knocks standing up, and IMO that particular incident was dorky, and they owned up to it publicly and then went on with their lives. IMO that was honorable and you were quite right to defend them. What RH is doing today is very different.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 11, 2008 3:05 UTC (Thu) by bojan (subscriber, #14302) [Link]

> Debian is part of SPI, which is a U.S. 501(c)3 non-profit corporation.

I reckon this part alone makes it different. Red Hat is a for-profit, publicly traded company. They turn over millions every year. SPI doesn't.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 11, 2008 6:02 UTC (Thu) by BrucePerens (subscriber, #2510) [Link]

<blockquote>I reckon this part alone makes it different. Red Hat is a for-profit, publicly traded company. They turn over millions every year. SPI doesn't.</blockquote>
That's really the problem, isn't it? Once the money (laws concerning stockholders, management concerns about losing customers or money) get in the way, you can't trust your distribution to do the right thing any longer.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 11, 2008 6:30 UTC (Thu) by bojan (subscriber, #14302) [Link]

I don't think that's a problem at all. By the nature of the relationship between Fedora and Red Hat, an undetected compromise of Fedora code can affect RHEL down the line. Hence, Red Hat have the responsibility to their shareholders even when it comes to this FOSS project they sponsor.

Money talks and all that... Nothing wrong with that.

All that being said, I'll bet $5 that if the root cause of the intrusion was something that required patching upstream (and it may be completely non-applicable), Red Hat folks would already be on it. After all, they run and live on open source software - why on earth would they want something like that go unpatched for the rest of us? After all, many of us are their customers.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 11, 2008 15:33 UTC (Thu) by jspaleta (subscriber, #50639) [Link]

As to the $5 bet:

http://skvidal.wordpress.com/2008/09/09/fedora-security-i...

"Just to dispell this concern. Every package we (fedora infrastructure) have installed or updated on a system since the incident occurred is public and available."

-jef

How can you be sure?

Posted Sep 11, 2008 16:04 UTC (Thu) by khim (subscriber, #9252) [Link]

I don't think that's a problem at all.

Puhlease.

I'll bet $5 that if the root cause of the intrusion was something that required patching upstream (and it may be completely non-applicable), Red Hat folks would already be on it.

How? "Here, please take this patch: we can't say why you should apply it and how it will work, but you SHOULD apply it" - like this? As others have said: they can't talk about things that went wrong, then don't want to talk about boundaries (what they can say or can not), etc. This will SEVERELY impede interaction with upstream. Will it make impossible to send the patch and get it accepted? Who knows. But I don't hold my breath...

After all, they run and live on open source software - why on earth would they want something like that go unpatched for the rest of us? After all, many of us are their customers.

The same logic applies to disclosure about current situation. They have used RHEL and Feedora systems, to handle that, right? They were setup by the best professionals available, right? And they STILL were compromised - so probably the same story can be repeated around the world again and again... Disclosure can hurt RedHat but will help customers - exactly as in situation with patch for upstream...

How can you be sure?

Posted Sep 11, 2008 22:32 UTC (Thu) by bojan (subscriber, #14302) [Link]

> we can't say why you should apply it and how it will work, but you SHOULD apply it

Say, for instance, it was the kernel problem. They can submit a patch that says: "such and such was fixed, which caused privilege escalation". This does not contain the information about the actual intrusion into their systems, but is a genuine patch with a genuine explanation.

> And they STILL were compromised

It doesn't necessarily follow that a security bug was the root cause of this.

The Fedora-Red Hat Crisis (Datamation)

Posted Sep 11, 2008 16:04 UTC (Thu) by smoogen (subscriber, #97) [Link]

Does that mean if one gets a paycheck, one can't be trusted either? Money is going to get in the way somewhere in that case..

So how does one pay for their food/shelter/net-access? Someone has to pay for it, and you are in some way 'in-debted' to them for that access, so are potentially non-trustable to do the right thing.

Copyright © 2012, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds