LWN: Comments on "Impact of the Debian OpenSSL vulnerability"

Impact of the Debian OpenSSL vulnerability

lysse — Thu, 22 May 2008 14:41:18 +0000

It's actually spelt correctly on a later occurrence... perhaps the author couldn't remember
and hedged their bets?

Impact of the Debian OpenSSL vulnerability

dvdeug — Tue, 20 May 2008 01:05:40 +0000

Oops... bad assumption. Apparently there's no direct connection between OpenSSL and OpenSSH;
Wikipedia states "Because of the prefix Open- on its name, OpenSSL is often associated with
OpenBSD; which distributes several programs using the naming style of Open*, like OpenSSH.
This is however a mistake as OpenSSL is developed completely outside of the scope of OpenBSD
by The OpenSSL Project".

Impact of the Debian OpenSSL vulnerability

nix — Mon, 19 May 2008 20:12:53 +0000

*Are* they a closely-related group? I thought the similarity of names was 
coincidence: but the similarity in... insular development styles does seem 
like it's stretching coincidence a bit.

Impact of the Debian OpenSSL vulnerability

dvdeug — Mon, 19 May 2008 10:32:37 +0000

But I think it a deep question; why is it the closely related group of OpenSSL and OpenSSH
developers who have this problem? Why is it that Debian developers can work hand in hand with
the developers on the X lists, on the GCC lists, on the kernel lists, but have trouble even
finding the OpenSSL list?

Impact of the Debian OpenSSL vulnerability

gdt — Mon, 19 May 2008 04:17:41 +0000

gettimeofday gives you more entropy than getpid.

Many certificates have an expiry date typically some whole years after generation. In those cases, gettimeofday() does not add entropy which cannot be calculated by an attacker.

Entropy for key generation has to be random, unpredictable and not influenced by external events. That's harder to find then you would hope, which is why even a kernel-based entropy collector doesn't produce much of it.

Using /proc/interrupts would not meet that requirement. At the least you would need to exclude interrupts from the network card (externally influenced) and from the quantum timer (predictable). Even then the attacker knows the value is monotonically increasing. The time of arrival of selected interrupts is a better source of entropy. But if you imagine the attacker holds an account on the same multiuser machine even that isn't too unpredictable.

In short, this is a problem which looks easy on the surface but is a nightmare when closely examined.

Impact of the Debian OpenSSL vulnerability

dvdeug — Sun, 18 May 2008 16:11:17 +0000

Virtually all of the code that most people run comes from the distribution. If no one is
looking at the distribution-local patches, then it is a failure of the many eyes concept. If
the distributions aren't sending their patches upstream, or the upstream is actively hostile
to the distributions, then it's a failure of the many eyes concept. Not that this would have
got noticed in a proprietary system until many systems got hacked, but that's no excuse.

Impact of the Debian OpenSSL vulnerability

muwlgr — Sun, 18 May 2008 06:49:33 +0000

gettimeofday gives you more entropy than getpid.
One could add contents of some /proc files to the mix (like: interrupts, meminfo, vmstat,
diskstats, ...).
No need to read unitialized data at all.

Impact of the Debian OpenSSL vulnerability

madscientist — Sat, 17 May 2008 22:17:34 +0000

I agree there is plenty of blame to go around: the Debian maintainer needs to accept some (if
you're maintaining a software package that is used by virtually the entire (F/OSS) world to
secure its computers, it behooves you to proceed with the most extreme caution and
circumspection possible), and the OpenSSL folks do too (a secret cabal mailing list is the
only reliable way to contact developers?!?!  Come on, that's lame.)

But to me the fundamental mistake here is EXACTLY the same as the recent OpenSSH 4.9->5.0
issue: the lack of upstream forwarding of patches.  To my mind it's an intrinsic
responsibility of every single downstream maintainer to forward upstream every patch they make
and every bug they receive, unless they are provably ONLY applicable to the distro and not the
base software.  I don't care if the developers are grumpy or mean or never accept patches
anyway or whatever: you still have to forward every single one and work with them as best you
can.  That also includes identifying yourself as the package maintainer for a given distro in
EVERY correspondence with upstream and making it clear when proposed changes are intended to
go into the distro proper.

IMO, if you can't sign up for that amount of work then you're not the right person to be
maintaining the package.  Especially not one like OpenSSL!

So, although I'm sure he's a great guy and I don't blame him for introducing the bug (everyone
makes mistakes!), in this case I do personally feel forced lay the bulk of the responsibility
for the debacle at the Debian maintainer's feet---for not forwarding the patch or working
effectively with upstream developers.  That's not to say I don't understand what happened or
I'm not sympathetic, because I do and I am.

I just hope everyone who maintains a package can learn from these recent problems and make a
renewed commitment to upstream developers--maybe we can make an adjustment to the distro
maintainer responsibilities documentation for each distro.  It would be great if something
positive was achieved as a result of this.

Impact of the Debian OpenSSL vulnerability

ikm — Sat, 17 May 2008 15:18:15 +0000

I'd say just not mess with the others' sources lightly. Some people like to come and say  oh
here, what the hell is this? Let's just cut it out! A story about a girl who tried to treat
her hamster for a "pimple" he had spurs to my mind.

Impact of the Debian OpenSSL vulnerability

pflugstad — Sat, 17 May 2008 12:10:51 +0000

Yes, this is exactly what happened.

OpenSSL has an internal random number generator. It uses MD5, SHA or some other hash (that's what the MD_Update function is that was commented out: Message Digests or hash function) over a pool of data.

When you first start up OpenSSL, and periodically during it's use, it/you add (or seed) more entropy to the pool by calling RAND_add. Typically, you'd call RAND_add, passing it a buffer of data from /dev/random or /dev/urandom. You keep doing this until until it got to a certain level of entropy. And then periodically call it again as entropy is used up.

Well, RAND_add maps to one of the functions that was patched by Debian to remove the MD_Update line that takes the provided buffer and stirs it into the pool. So basically, the only entropy left in the pool are basic things the OpenSSL like the process id.

"Many Eyes" still wins

PO8 — Sat, 17 May 2008 08:11:46 +0000

What makes you think this bug *ever* would have been found by anyone other than particularly
clever miscreants in a proprietary system?  There are few people looking for similar bugs in
closed SW, and even fewer that have any motivation to do anything other than conceal these
bugs when they find them.

As far as I know, this bug was finally found when Bello noticed a bad patch had been applied,
and was able to diagnose the resulting weakness by inspecting the source.  Other than that,
"many eyes" hasn't helped at all here...

Impact of the Debian OpenSSL vulnerability

BackSeat — Sat, 17 May 2008 07:03:40 +0000

a clueless guy decided to mess with code he didn't understand.

So what should this "clueless guy" do when he wants to make a change? He should have checked with the OpenSSL folk to ask if there was a problem in making the change. Which is exactly what he did. You may want to read the other LWN articles on this vulnerability: no one is saying that the Debian maintainer is innocent, but he certainly isn't the only one who should accept some responsibility for what happened.

Impact of the Debian OpenSSL vulnerability

ianburrell — Sat, 17 May 2008 02:35:37 +0000

OpenSSL does use /dev/random for entropy if it is available.  My guess is that goes through
the interface that was commented out.  The security hole is that someone removed the hook that
entered all buffers into the entropy pool while trying to just remove unitialized buffers.  So
/dev/random was being used but it didn't do any good.

Impact of the Debian OpenSSL vulnerability

Miravlix — Sat, 17 May 2008 02:29:47 +0000

You could read the story. The problem is precisely that the many eyes wasn't used and a clueless guy decided to mess with code he didn't understand. Mistakes made here: 1: Coder without understanding the code messed with it. 2: Code wasn't applied so the "Many Eyes" concept was used to find the horrible flaw.

Impact of the Debian OpenSSL vulnerability

freethinker — Sat, 17 May 2008 01:32:48 +0000

I find it quite revealing that it took the "many eyes" supposedly possessed by the free
software community over a year and a half to notice such a serious bug - and this in Debian,
one of the most widely used distributions.

Time we buried the "many eyes" myth once and for all.

Impact of the Debian OpenSSL vulnerability

rfunk — Fri, 16 May 2008 21:15:17 +0000

OpenSSL supports systems that don't have /dev/random, but provides the 
same API to everyone.  So when /dev/random is available, it's used as a 
seed for OpenSSL's pseudo-random number generator, rather than being used 
directly.  If they used /dev/random directly they'd need totally separate 
code paths depending on whether it was available.

I wonder how the OpenBSD folks handle it.....

Impact of the Debian OpenSSL vulnerability

bronson — Fri, 16 May 2008 21:08:05 +0000

A PID offers so little additional entropy that it's basically worthless.  Still, it can't hurt
to include it, right?

Dunno about thiat...  If the PID weren't mixed into the randomness, this vulnerability would
have been found within days if nothours.  The slight additional complexity of mixing the PID
in managed to hide a massive security problem for two years.

So, if /dev/random is good enough, perhaps mixing in a tiny amount more entropy ends up being
more harmful than helpful.  It seems to have been in this case.

Impact of the Debian OpenSSL vulnerability

riddochc — Fri, 16 May 2008 20:21:26 +0000

I've been following this here on LWN, but haven't really looked too deep at the technical details yet. I somehow got the impression that the kernel crowd has figured out entropy sources adequate for providing for the needs of both security-sensitive (/dev/random) and less sensitive (/dev/urandom) software.

I'm curious about why OpenSSL considers /dev/random insufficient? Is it in fact insufficient? Can somebody provide a link about this?

Impact of the Debian OpenSSL vulnerability

kmccarty — Fri, 16 May 2008 20:14:25 +0000

correcting myself slightly...

it appears that they have not yet generated (or at least not made available) keys that would
have been created by 64-bit or big-endian systems, but those would only be a small subset of
the total number of Debian-generated keys out in the wild.

Impact of the Debian OpenSSL vulnerability

kmccarty — Fri, 16 May 2008 20:09:07 +0000

Yes, and it is already done, see reference 3 of the article.

Impact of the Debian OpenSSL vulnerability

johnkarp — Fri, 16 May 2008 19:58:43 +0000

My impression is that there is rarely any single good source of entropy, 
so it must be scrounged up from as many minor sources as possible... 
hardware interrupt timings, mouse movements, etc. The PID probably only 
adds a bit or two, but every bit helps.

Impact of the Debian OpenSSL vulnerability

jengelh — Fri, 16 May 2008 19:09:09 +0000

All ur keyz r belong to us.so much short and to the point.

Impact of the Debian OpenSSL vulnerability

welinder — Fri, 16 May 2008 19:07:09 +0000

Why is the pid used for this at all?  There is practically no entropy
in it, so the effect of using it is to hide deficiencies in the proper
entropy sources.

Impact of the Debian OpenSSL vulnerability

clugstj — Fri, 16 May 2008 19:03:05 +0000

If the only source of entropy was the PID of the process generating the key, wouldn't it be
easy to produce a list of ALL of the weak keys?

Impact of the Debian OpenSSL vulnerability

Thue — Fri, 16 May 2008 18:43:07 +0000

Perhaps it is leet speak like warez, passwordz, etc... (just kidding :) )

Impact of the Debian OpenSSL vulnerability

jengelh — Fri, 16 May 2008 17:47:14 +0000

compromised, even in AE.