OpenBSD IPSEC backdoored?

[Posted December 15, 2010 by corbet]

From:		Theo de Raadt <deraadt-AT-cvs.openbsd.org>
To:		tech-AT-cvs.openbsd.org
Subject:		Allegations regarding OpenBSD IPSEC
Date:		Tue, 14 Dec 2010 15:24:39 -0700
Message-ID:		<201012142224.oBEMOdWM031222@cvs.openbsd.org>
Archive‑link:		Article

I have received a mail regarding the early development of the OpenBSD
IPSEC stack.  It is alleged that some ex-developers (and the company
they worked for) accepted US government money to put backdoors into
our network stack, in particular the IPSEC stack.  Around 2000-2001.

Since we had the first IPSEC stack available for free, large parts of
the code are now found in many other projects/products.  Over 10
years, the IPSEC code has gone through many changes and fixes, so it
is unclear what the true impact of these allegations are.

The mail came in privately from a person I have not talked to for
nearly 10 years.  I refuse to become part of such a conspiracy, and
will not be talking to Gregory Perry about this.  Therefore I am
making it public so that
    (a) those who use the code can audit it for these problems,
    (b) those that are angry at the story can take other actions,
    (c) if it is not true, those who are being accused can defend themselves.

Of course I don't like it when my private mail is forwarded.  However
the "little ethic" of a private mail being forwarded is much smaller
than the "big ethic" of government paying companies to pay open source
developers (a member of a community-of-friends) to insert
privacy-invading holes in software.

----

From: Gregory Perry <Gregory.Perry@GoVirtual.tv>
To: "deraadt@openbsd.org" <deraadt@openbsd.org>
Subject: OpenBSD Crypto Framework
Thread-Topic: OpenBSD Crypto Framework
Thread-Index: AcuZjuF6cT4gcSmqQv+Fo3/+2m80eg==
Date: Sat, 11 Dec 2010 23:55:25 +0000
Message-ID: <8D3222F9EB68474DA381831A120B1023019AC034@mbx021-e2-nj-5.exch021.domain.local>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Status: RO

Hello Theo,

Long time no talk.  If you will recall, a while back I was the CTO at
NETSEC and arranged funding and donations for the OpenBSD Crypto
Framework.  At that same time I also did some consulting for the FBI,
for their GSA Technical Support Center, which was a cryptologic
reverse engineering project aimed at backdooring and implementing key
escrow mechanisms for smart card and other hardware-based computing
technologies.

My NDA with the FBI has recently expired, and I wanted to make you
aware of the fact that the FBI implemented a number of backdoors and
side channel key leaking mechanisms into the OCF, for the express
purpose of monitoring the site to site VPN encryption system
implemented by EOUSA, the parent organization to the FBI.  Jason
Wright and several other developers were responsible for those
backdoors, and you would be well advised to review any and all code
commits by Wright as well as the other developers he worked with
originating from NETSEC.

This is also probably the reason why you lost your DARPA funding, they
more than likely caught wind of the fact that those backdoors were
present and didn't want to create any derivative products based upon
the same.

This is also why several inside FBI folks have been recently
advocating the use of OpenBSD for VPN and firewalling implementations
in virtualized environments, for example Scott Lowe is a well
respected author in virtualization circles who also happens top be on
the FBI payroll, and who has also recently published several tutorials
for the use of OpenBSD VMs in enterprise VMware vSphere deployments.

Merry Christmas...

Gregory Perry
Chief Executive Officer
GoVirtual Education

"VMware Training Products & Services"

540-645-6955 x111 (local)
866-354-7369 x111 (toll free)
540-931-9099 (mobile)
877-648-0555 (fax)

http://www.facebook.com/GregoryVPerry
http://www.facebook.com/GoVirtual

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 4:02 UTC (Wed) by abadidea (guest, #62082) [Link] (1 responses)

Anyone interested in a community auditing effort, check out the wiki set up by #openbsd http://pohl.ececs.uc.edu/opendoku/doku.php?id=start

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 18:22 UTC (Wed) by utoddl (guest, #1232) [Link]

That server is not responding at the mo. Hmm.

Strange SSH flaw raised by French porter

Posted Dec 15, 2010 6:15 UTC (Wed) by boog (subscriber, #30882) [Link] (6 responses)

Along similar lines, I remember a flaw in SSH reported by the French porter of the software who produced a version with a weaker cryptographic key to comply with French law at the time. I'm not competent to judge the reality of the problem, but he certainly thought it was a backdoor.

From:

ftp://ftp.nluug.nl/pub/metalab/docs/linux-doc-project/lin...

I often like to point out an incomprehensible weakness of the protocol concerning the "padding" (known as covered channel): in both version 1 and 2 the packets, have a length which is a multiple of 64 bits, and are padded with a random number. This is quite unusual and therefore sparing a classical fault that is well known in encrypting products: a "hidden" (or "subliminal") channel. Usually , we "pad" with a verified sequence as for example, give the value n for the byte rank n (self describing padding). In SSH, the sequence being (by definition) randomized, it cannot be checked. Consequently, it is possible that one of the parties communicating could pervert / compromise the communication for example used by a third party who is listening. One can also imagine a corrupted implementation unknown by the two parties (easy to realize on a product provided with only binaries as generally are commercial products). This can easily be done and in this case one only needs to "infect" the client or the server. To leave such an incredible fault in the protocol, even though it is universally known that the installation of a covered channel in an encryption product is THE classic and basic way to corrupt the communication, seems unbelievable to me . It can be interesting to read Bruce Schneier's remarks concerning the implementation of such elements in products influenced by government agencies. (http://www.counterpane.com/crypto-gram-9902.html#backdoors).

I will end this topic with the last bug I found during the portage of SSH to SSF (French version of SSH), it is in the coding of Unix versions before 1.2.25. The consequence was that the random generator produced ... predictable... results (this situation is regrettable in a cryptographic product, I won't go into the technical details but one could compromise a communication while simply eavesdropping). At the time SSH's development team had corrected the problem (only one line to modify), but curiously enough without sending any alert, not even a mention in the "changelog" of the product... one wouldn't have wanted it to be known, he wouldn't have acted differently. Of course there is no relationship with the link to the above article.

Strange SSH flaw raised by French porter

Posted Dec 15, 2010 12:41 UTC (Wed) by clump (subscriber, #27801) [Link] (3 responses)

I think you're illustrating the issues surrounding the SSH monoculture. There are very few alternatives to OpenSSH.

Strange SSH flaw raised by French porter

Posted Dec 15, 2010 15:24 UTC (Wed) by nix (subscriber, #2304) [Link]

Quite so.

"Error in complex cryptographic system -> conspiracy" is not exactly ironclad logic.

(I had no *idea* that there were so very many conspiracies around. There must be millions of them. Has anyone ever implemented *any* complex cryptographic system that didn't have at least one horrible flaw detected in hindsight? This is why Schneier, among others, no longer calls for better cryptography as the cure for all ills: when you get down to it any cryptosystem must be designed and implemented by human beings, and then deployed by others who know much less about the cryptosystem than the designers or implementors, and human beings make mistakes, or don't care, or have perverse incentives leading them to do a worse job than they might otherwise even absent conspiracy. So this is all a huge human-factors problem in the end, like virtually everything else.)

Strange SSH flaw raised by French porter

Posted Dec 16, 2010 8:28 UTC (Thu) by tzafrir (subscriber, #11501) [Link]

There's a well-known alternative to the client: Putty:
http://www.chiark.greenend.org.uk/~sgtatham/putty/

I believe it is the most common client on win32. It is also works on Linux:
http://packages.debian.org/sid/putty-tools

As for the server: I know that dropbear is popular on embedded systems:
http://matt.ucc.asn.au/dropbear/dropbear.html

There are a number of other free implementations.

Strange SSH flaw raised by French porter

Posted Dec 16, 2010 20:47 UTC (Thu) by Comet (guest, #11646) [Link]

The version number cited and the description suggests that this relates to a port of the original SSH, not OpenSSH.

Strange SSH flaw raised by French porter

Posted Dec 16, 2010 3:44 UTC (Thu) by djm (subscriber, #11651) [Link] (1 responses)

That criticism applies to any protocol that uses non-determinism. By that logic, DH, CBC ciphers and anything that uses random challenges or nonces is flawed.

It only makes sense if the endpoints have no other way of leaking bits, which is not the case for the overwhelming majority of systems.

Strange SSH flaw raised by French porter

Posted Dec 17, 2010 12:05 UTC (Fri) by tialaramex (subscriber, #21167) [Link]

Leaking bits on the same channel has advantages in convenience and discreetness. It's convenient because when you tap the channel (perhaps under cover of claiming you'll do traffic analysis) you get this leaked data without also needing to tap a separate channel.

It's discreet because a fairly competent analysis of the affected system will find a separate channel, an IPSec implementation that sends mysterious ICMP packets to an anycast address will cause the researcher to go look up who owns that anycast address, what it's for, what's in the ICMP packets, etc. Whereas if the data is just in some padding bytes that's not something you'd necessarily do more than glance at.

Also there have been weaknesses involving one of the parties in a two party protocol choosing a bad "random" challenge or nonce. The protocol will usually include a step to detect this and retry, which will be seamless unless bad guys are involved, but of course an implementation which does not check will appear to work - it's just not secure.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 11:06 UTC (Wed) by AlexHudson (guest, #41828) [Link] (1 responses)

I realise that the immediate geek response is to worry about the code, but surely the bigger story - if this was true - was that the FBI is allegedly monitoring the traffic of the Executive Office of the United States?!

And is the EOUSA really the parent organisation of the FBI? That sounds weird to European ears.

US _Attorneys_

Posted Dec 15, 2010 12:55 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

Wikipedia says that the FBI reports to the DoJ, and EOUSA is the Executive Office for United States Attorneys which is a DoJ department.

So that's _Attorneys_ not _America_

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 11:22 UTC (Wed) by MisterIO (guest, #36192) [Link]

More importantly, is Openbsd's IPSEC code used in any part of current Linux kernels?

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 13:17 UTC (Wed) by clugstj (subscriber, #4020) [Link] (8 responses)

It would be truly astounding if someone could put such a subtle back door in the code that hadn't been discovered in 10 years - and ported to other platforms.

Is anyone so paranoid (except Theo) that they believe this is really a problem?

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 14:15 UTC (Wed) by foom (subscriber, #14868) [Link] (3 responses)

I dunno, I find it feasible. Software has bugs. Even 10-year-old bugs. If you can deliberately insert the right kind of bug in the right place...

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 16:58 UTC (Wed) by whitemice (guest, #3748) [Link] (2 responses)

Sure there are bugs ... and then there is a significant *feature* [which is what this would technically be] of a protocol stack. You aren't going to mistake something like this for a typo.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 18:02 UTC (Wed) by marcH (subscriber, #57642) [Link]

A large number of honest, unintentional bugs in C end up being security risks. For sure a good and dedicated team of engineers can come up with security risks that look like bugs.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 21:57 UTC (Wed) by njs (subscriber, #40338) [Link]

Have you seen the Underhanded C contest? Some of the entries are pretty amazing.

http://underhanded.xcott.com/

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 14:27 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

It's pretty obvious that Theo is skeptical, that's exactly why he has posted this. If he discards it, and it's later found (astoundingly) to be true, he looks guilty. If he had instantly thought "this must be true" he'd have presumably ordered a quiet audit of IPSEC to try to find the issue before making headlines.

The story sounds very strange, but to be fair lots of true stories are stranger. Named developers and a specific period are given, if these clues aren't enough for us to find any evidence, then I agree that there was probably none to find. The supposed target (and the fact that people involved who didn't need to know claim they were told it) is a weird choice too, but it's not as though the US Attorneys (basically people who prosecute federal crimes) are above suspicion themselves.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 15:25 UTC (Wed) by nix (subscriber, #2304) [Link]

Quite. I mean, it's a bug in IPSEC. Even if IPSEC turns out to be broken completely from top to bottom, it's used so little that it's not going to rock anyone's boat at all.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 16:56 UTC (Wed) by whitemice (guest, #3748) [Link]

+1 It isn't feasible at all. Something as complex as an IPSec backdoor [and IPSec itself is *complicated*] would be obvious as heck; I strongly doubt the potential for a "subtle" back door.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 17:32 UTC (Wed) by iabervon (subscriber, #722) [Link]

It's entirely possible for a backdoor to look like a regular bug of no particular consequence. For example, having the second and third arguments to memset backwards is a common mistake that doesn't usually cause crashes or corruption but may leak data, and usually isn't particularly security-critical either (because the leaked data doesn't end up going anywhere anyway). So there are a lot of memset bugs that have been made and fixed over the past ten years, and people fixing them ordinarily don't consider whether they might have been intentionally introduced to leak key material to observers. So, even if no backdoors were found, and even if reviewers would find any flaw in the code in that period, it doesn't mean that there weren't backdoors.

Obviously, people frequently find security flaws that were accidental, had been there for a while, and could be exploited with detailed knowledge. Among these, one that was intentional but of a common form wouldn't stand out.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 14:24 UTC (Wed) by ber (subscriber, #2142) [Link] (2 responses)

http://blogs.csoonline.com/1296/an_fbi_backdoor_in_openbsd reports more info from Gregory Perry.

OpenBSD IPSEC backdoored?

Posted Dec 15, 2010 15:29 UTC (Wed) by MisterIO (guest, #36192) [Link]

That article gives Perry's claims much more substance!

OpenBSD IPSEC backdoored?

Posted Dec 16, 2010 2:18 UTC (Thu) by gdt (subscriber, #6284) [Link]

That article links to a denial from one of the "participants", who states he has never worked on any FBI-related matter, contradicting Mr Perry's claim.

blog.scottlowe.org "Allegations regarding FBI involvement with OpenBSD"
Scott Lowe, 2010-12-14

Includes some very fine comments admiring the "well you would say that either way" paradox.

Sneaky way to get publicity ?

Posted Dec 15, 2010 16:18 UTC (Wed) by abacus (guest, #49001) [Link] (1 responses)

Maybe this message is just a sneaky way of Gregory Perry to advertise his own business ?

Sneaky way to get publicity ?

Posted Dec 15, 2010 17:39 UTC (Wed) by danieldk (guest, #27876) [Link]

At the very least, it has some plausibility.

Remember that Debian's OpenSSL was broken for two years without anyone noticing. If some package maintainer intentionally introduced such a bug, would anyone notice?

Proprietary software is even worse, since no third party can check the source. But it is still a viable attack route in free software.

Bad, bad, bad...

Posted Dec 15, 2010 22:47 UTC (Wed) by proski (guest, #104) [Link] (14 responses)

Publishing private e-mails is bad. Theo de Raadt should have asked permission first.

Bad, bad, bad...

Posted Dec 15, 2010 23:14 UTC (Wed) by hingo (guest, #14792) [Link]

That was my first reaction too.

Bad, bad, bad...

Posted Dec 15, 2010 23:25 UTC (Wed) by daney (guest, #24551) [Link] (3 responses)

He did state his justification for publishing it. So perhaps a better question is not whether you think it is 'Bad, bad, bad...', but what you think about his stated justification.

Bad, bad, bad...

Posted Dec 16, 2010 4:17 UTC (Thu) by proski (guest, #104) [Link] (2 responses)

At least Theo should have asked for permission. He should have stated his intentions privately. His correspondent should have been given a chance to object and to give his arguments against the publication. I see no evidence that Theo has done it.

Bad, bad, bad...

Posted Dec 16, 2010 8:46 UTC (Thu) by renox (guest, #23785) [Link] (1 responses)

Bah, even if the correspondent had refused, Theo would still have published the email: he explains why in his email and his reasons are valid IMHO.

Bad, bad, bad...

Posted Dec 17, 2010 10:42 UTC (Fri) by marcH (subscriber, #57642) [Link]

> Bah, even if the correspondent had refused, Theo would still have published the email

This would still have been more polite.

Bad, bad, bad...

Posted Dec 16, 2010 3:31 UTC (Thu) by djm (subscriber, #11651) [Link] (7 responses)

Publishing a private email is worse than letting users know of a potential backdoor? Remind me never to use software you maintain

Bad, bad, bad...

Posted Dec 16, 2010 4:24 UTC (Thu) by proski (guest, #104) [Link] (3 responses)

It's obsolete wireless drivers (orinoco*, spectrum_cs and MadWifi), and I'll appreciate if you use modern drivers instead (hostap, ath5k and ath9k). You'll also need to avoid cvsutils. If you want to be really careful, you'll need to avoid Midnight Commander and GNU GRUB. Even though I don't maintain them anymore, I was their comaintainer in the past. Sorry for causing so much trouble to you. I don't think I'll ever accept maintenance of another free software project. It takes too much effort, and the only appreciation I get is the comments like yours.

Bad, bad, bad...

Posted Dec 16, 2010 4:41 UTC (Thu) by JoeBuck (guest, #2330) [Link] (1 responses)

Thanks for all your contributions to code that I have used over the years. However, I still think you are wrong in this case. If you really want to claim that "don't publish mail without permission" is an ironclad rule, you risk creating perverse situations. If Theo writes back, and the author of the accusation says "no, you can't share this", what is he supposed to do? Do all the security audits all by himself?

Sorry, but "off the record" is something that has to be agreed to by both parties, it can't be imposed by only one. If you make a blockbuster accusation by private mail, you can expect that accusation to be shared, if only to confirm whether it is true or false.

There are many possibilities: the accuser could be making it up, he could be wrong, or maybe there was an effort to insert back doors that wasn't followed through on, or maybe the back door code was inserted but no longer exists because the relevant code has been rewritten, or maybe there still is a back door. But the only way to determine the truth is for developers to investigate, and even if an effort is made to contain such an explosive charge, it's likely to leak.

Bad, bad, bad...

Posted Dec 16, 2010 16:55 UTC (Thu) by cry_regarder (subscriber, #50545) [Link]

"Don't post private emails" doesn't equal "off the record". He could have paraphrased the original email.

Bad, bad, bad...

Posted Dec 17, 2010 17:07 UTC (Fri) by clump (subscriber, #27801) [Link]

Firstly, thank you for your contributions. Many people feel the way I do, whether we say so or not. Secondly, I agree that publishing the email without consent was inappropriate. If permission wasn't granted, the message could have been paraphrased as has been suggested.

There's a culture surrounding OpenBSD that attacks anyone questioning the project's behaviour. The Broadcom wireless driver fiasco is a perfect example.

Bad, bad, bad...

Posted Dec 16, 2010 5:27 UTC (Thu) by dlang (guest, #313) [Link]

telling people that you have been informed via private e-mail that there is a backdoor, and even paraphrasing what you were told does not violate privacy expectations by publishing private e-mail, and is just as effective in letting people know that there is a problem.

Bad, bad, bad...

Posted Dec 16, 2010 7:48 UTC (Thu) by lkundrak (subscriber, #43452) [Link]

This had been of absolutely no use and no value to the users, except for FUD.

It basically just says "Your system is backdoored, or not. We do not have a fix, so you may need to stop using this, or maybe keep using it if it's not backdoored". There's something most vendors that care about security of their users follow called responsible disclosure. That means that you disclose a security issue only when you're sure that all vendors that ship the code had a chance to provide their users with a fix at the time the issue is made public.

Theo should definitely first investigate the issue himself so that he could provide a more valuable response (or refrain from saying anything at all if the issue turns bogus).

His motivation obviously was to publicly offend the guy, publishing his private e-mail and demonstrating that he doesn't trust him. I hope he's not stupid enough to publish an issue that he thinks could be real like this, but he definitely enjoys being a dick.

Bad, bad, bad...

Posted Dec 16, 2010 14:21 UTC (Thu) by dunlapg (guest, #57764) [Link]

I think the main thing to think about, from a policy perspective, it this: Theo has cut himself (and potentially others) off from a potential source of information: namely, people who know about a potential security issue, but doesn't want to be fingered as revealing it; who would prefer to keep the issue itself a secret rather than risk exposing his own identity. Those people will now not mail Theo (or potentially anyone) for fear of being exposed.

Ethical gray area

Posted Dec 17, 2010 11:16 UTC (Fri) by pr1268 (guest, #24648) [Link]

I take it from the tone of this discussion that Theo's actions are treading an ethical gray area. There's obviously good arguments on either side of whether what he did was appropriate.

I personally applaud him for exposing even the mere possibility of a back door (thereby likely deploying developers and hackers to thoroughly examine the code), but I also agree that he may have been wise to ask Perry's permission to post the e-mail verbatim.

Just my observation from the gallery, and my $0.02.

OpenBSD IPSEC backdoored?

Posted Dec 16, 2010 21:42 UTC (Thu) by jd (guest, #26381) [Link]

The question of whether or not there is a backdoor is obviously the critical question to ask. Or, rather, part of the critical question. You also need to ask WHICH IPSec stack was allegedly attacked and precisely when.

As best as I recall, there were three *BSD IPSec stacks in the very early days. I don't know if OpenBSD directly used one of these or developed their own. I know that some systems used OpenBSD's stack later on, but in the very early days there may well have been a fair bit of flow from OpenBSD into these more basic prototypes, which would broaden the collateral damage.

At the same time, the earlier the backdooring is supposed to have taken place, the more likely that some branches will have lost said backdoors (some through auditing, others through optimizations). There's a LOT of vendors out there with "secure" BSD appliances that could be considered by the trade press as potentially tainted unless they can prove otherwise. Those vendors might well want to establish that their systems are as safe as claimed - or, if they would rather be discrete, establish if any fixes are necessary and smuggle them over to OpenBSD.