LWN: Comments on "Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)"

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

robbe — Fri, 17 Feb 2012 14:13:45 +0000

That something is wrong with the CA industry is not a particularly original observation.

As to common practise, I cannot seem to find anything right now, but a company we dealt with in the past is reselling certificates of various vendors, and the offers on their webpage include "root signing", which I understand to be signing of an intermediate CA. Prices for that begin at 10 k€. These seem to be offered for Comodo, RapidSSL and Thawte.

Not everyone is that open with their offers, but I'm pretty sure these are available from many CAs for an appropriate price. The legitimate(?) use case is a company that would rather pay a lump sum to have their MITM device work painlessly, than have to install their self-signed MITM-CA certificate as trusted on every end-user device.

Oh, and Symantec is selling gateways for anti-virus and data leakage prevention. Both purposes would be served well by a SSL-MITM, and probably have one built in (it's state of the art). A CA certificate signed by a generally accepted root CA (like Verisign's) would go fine with that. I am not aware of them doing that.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

smcv — Sun, 12 Feb 2012 14:02:40 +0000

When used correctly, it's a security feature. CAs keep the long-lived key that the browser trusts (root key) offline, and sign certificates with a short-lived intermediate CA signed by the root key. They only need to take the root key out of storage when the intermediate CA is close to expiring.

The difference here is that Trustwave gave an intermediate CA key to another company rather than keeping control of it themselves.

Mozilla and Certificate Authorities

dark — Sun, 12 Feb 2012 12:38:40 +0000

I recently started using Certificate Patrol (firefox add-on) and it notifies me whenever it accepts a new certificate and gives me a chance to inspect the details. After the first day I had all my usual sites in there so I'm mostly browsing without interruptions again.

Since I left all the CAs at their default, I can easily tell the difference between a site signed by a possibly-dodgy CA and one that doesn't have a valid signature at all. In the first case I get a notification from Certificate Patrol and I can reject it if it looks too odd for the site. In the second case I get the usual Firefox warning.

This is in addition to Certificate Patrol's main feature, which is to warn me if a site's certificate changes unexpectedly. The kind of snooping that's described in this article will make it look like many sites' certificates have changed and that will set off all the warning bells.

Maybe this approach will work for you too?

Mozilla and Certificate Authorities

lambda — Sun, 12 Feb 2012 06:30:55 +0000

You can permanently disable certificates in Firefox. I have done so myself after previous incidents, like Comodo or DigiNotar. You will still have the certificates there, but you can mark them as untrusted.

The problem is, this breaks the web. Unless everyone does it, thus forcing websites to switch certificate providers, it mainly just frustrates you by making you jump through more hoops any time you go to a site signed with one of those certificates. There is no good way to distinguish between "I'm being MITM'd by a compromised cert" and "this website is still using one of the CAs that I don't trust." And while you can click through the warning if the main page is signed by an untrusted CA, it's really hard to fix the problem when it is using resources (images, JavaScript, and CSS) signed by an untrusted CA while the main content is signed by a trusted one; then you don't get a big scary screen to click through and say "yes, this is OK," you just don't get any of the JavaScript or CSS working and have to dig through the site to try and figure out which resources use what CAs and selectively re-enable those.

I've had to go back and re-trust most of the certificates that I had un-trusted, because I just couldn't use Firefox that way.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

acolin — Sun, 12 Feb 2012 05:17:14 +0000

> Not just speculation as this is exactly what the company I work for does. They claim
Same here. Not sure what's their claim. They setup the MIM recently and without any announcement (afaik). I'm sure there's something in the contract that effectively 'pre-approved' such action, but it's still not nice, because effectively it's secret surveillance until you check (how often does that happen?) or somebody points it out to you. Luckily not everybody was locked into IE and the employees who happened to use browsers without the private cert (yet) kindly exposed the scheme.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

dlang — Fri, 10 Feb 2012 19:02:02 +0000

You are mistaken, it is very possible to have multiple certs issued for the same name, the checking that's done is to check that the cert is signed by someone you trust (or is signed by someone who's signed by someone.. who's signed by someone you trust)

The same company can sign multiple certs, or you can have different companies sign certs all for the same name.

This is not just some theoretical use either, this is exactly what's done to replace the certs that expire (typically every year), the vendor signs a new cert for the company, and for some time both the old and new certs are valid. During that time the company replaces the old cert with a new cert on the servers and everyone just accepts the new cert.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

raven667 — Fri, 10 Feb 2012 19:01:16 +0000

That's not how it works at all, your browser does not have any finger print on record for any certificate. Certificates can change at will as long as the commonName matches the DNS name and it's ultimately signed by a root you trust, that's all that is verified. There is nothing preventing you from having many legit certs with the same name.

There are proposals for certificate pinning which will store fingerprints much like SSH does, I believe Chrome now does this for *.google.com certs by default, but this is not standard or required or widely deployed behavior.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

jd — Fri, 10 Feb 2012 18:49:26 +0000

It is certainly possible I'm missing something, because it has always been my understanding that browsers are supposed to verify that the certificate fingerprint matches the fingerprint on record whether or not the root certificate is approved of. That the only way to replace a certificate during its lifetime without generating an error (even with an approved root) was to revoke the prior certificate first.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

Aissen — Fri, 10 Feb 2012 13:56:17 +0000

There's no need for that. The public key of that certificate and an example of forged sub-certificate for, say gmail.com ought to be enough proof.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

erwbgy — Fri, 10 Feb 2012 13:43:42 +0000

You just need to issue yourself a CA cert and place that certificate into the list of trusted certs on your computers. That's not hard to do with any OS. All that's required at that point is to proxy all the HTTPS traffic originating from the network and generate SSL certs for the destinations on the fly.

Not just speculation as this is exactly what the company I work for does. They claim it is to enable them to detect malware hidden in SSL connections and that they exclude sensitive sites like banks.

Mozilla and Certificate Authorities

njwhite — Fri, 10 Feb 2012 10:57:58 +0000

You may be interested in a blog post by one of the Tor guys about how he does exactly that: https://blog.torproject.org/blog/life-without-ca

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

ekj — Fri, 10 Feb 2012 10:37:50 +0000

This fails to work in many jurisdictions. Informing the employee is not sufficient. The problem is that I can easily refrain from using the company-network for private stuff myself.

But I have no control over what others do. Friends can (and sometimes do) stumble across my company-email somehow, somewhere, then use that for sending me something private. There's no way for me to completely guard against this possibility. You could argue that they "should" know better, but that's not enough to get you out of potential legal problems.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

mpr22 — Fri, 10 Feb 2012 09:37:06 +0000

I guess "a strict subset of your compulsory lunch break, in a separate office with the blinds closed and your monitor facing away from the door" might qualify.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

farnz — Fri, 10 Feb 2012 09:02:04 +0000

The thing that makes this obnoxious is that you can get policy enforcement with an internal-only CA - it's just that you have to get clients to accept that the CA chain is broken if you cannot install the internal CA certificate on them.

Breaking the supposed identity guarantees of SSL for the benefit of one company's monitoring system is a bad move - what would have happened if (for example) that company had turned out to be a hotel chain, using it to snoop on visitors' use of private e-mail and the like?

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

slashdot — Fri, 10 Feb 2012 02:25:03 +0000

For instance, by offering a high reward ($10-1000k) to anyone providing the private key of a certificate that can sign trusted certificates for any domain.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

slashdot — Fri, 10 Feb 2012 02:23:23 +0000

How about looking for those other CAs and removing them too?

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

slashdot — Fri, 10 Feb 2012 02:13:30 +0000

How about Mozilla setting up DNSSEC-enabled DNS servers, and getting rid of all non-Mozilla CAs when that DNS server is in use?

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

martin.langhoff — Fri, 10 Feb 2012 01:49:44 +0000

I want an SSL stack that fully ignores these subordinate certs, unless I click on the "Allow MITM" checkbox.

Who invited this misfeature to the party?

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

elanthis — Fri, 10 Feb 2012 01:33:34 +0000

... what is an ordinate amount of time to surf pr0n at work?

Mozilla and Certificate Authorities

ras — Thu, 09 Feb 2012 23:54:40 +0000

> I am dismayed that Mozilla Firefox does not allow one to permanently disable the built-in certificates.

It does. At least Firefox 4.0 on Debian does. Clicking "Edit Trust" allows you to stop trusting the cert for various things, such as "Identifying web Sites". Those settings are remembered across reboots.

You are correct in saying if you delete a cert it re-appears at the next restart, but it reappears with all trust settings disabled.

Mozilla and Certificate Authorities

signbit — Thu, 09 Feb 2012 23:31:48 +0000

Try it. They are embedded in a shared library.

Mozilla and Certificate Authorities

job — Thu, 09 Feb 2012 23:22:57 +0000

Are you sure Firefox restores them? That sounds strange. Perhaps you are just not privileged enough, and Firefox gives no error message?

Mozilla and Certificate Authorities

signbit — Thu, 09 Feb 2012 22:58:40 +0000

I am dismayed that Mozilla Firefox does not allow one to permanently disable the built-in certificates.

I'd like to have a browser that I start with a clean slate in terms of trusted certificate authorities. I have no idea who "TÜRKTRUSTElektronikSertifikaHizmetSağlayıcısı" is or why should I trust anything they sign.

You can delete stuff from Firefox, but it gets restored after you restart the browser. Not good!

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

raven667 — Thu, 09 Feb 2012 22:18:41 +0000

This reminds me of a story I heard about Network General, I believe. The admins were doing some network troubleshooting and in the process discovered that one of the executives was spending an inordinate amount of time surfing pr0n at work. They brought this up to their supervisor after having been exposed to it and their supervisor sent out an all-staff mail to the effect of "We have the highest concentration of Sniffer(tm)s on the planet, please keep this in mind when using the network." Amazingly the traffic stopped. 8-)

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

raven667 — Thu, 09 Feb 2012 22:09:32 +0000

> You don't need a subordinated root of a trusted CA to do it.

Its true that the traditional way of setting up a DLP/firewall/SSL proxy is to use an internal CA that is trusted by the clients, I can only imagine that the customer didn't want the administrative overhead of touching every machine to load certs or had some clients they couldn't touch that they still needed policy enforcement on. Signing a subroot which will be trusted by the majority of clients is a technically easy way around this but clearly even Trustwave agrees that this is a bad idea which is why they have very publicly stopped doing it.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

dlang — Thu, 09 Feb 2012 22:00:32 +0000

they aren't allowed to secretly monitor users, but as a result of lawsuits over admins seeing private things while doing normal system maintinance of mail servers (as an example), it's standard boilerplate for the HR documents to say that the company has the right to monitor your actions on their equipment (including their network if they allow you to use your personal equipment on it)

so it's lost in the HR boilerplate but it's not secret.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

gmaxwell — Thu, 09 Feb 2012 21:28:12 +0000

"a company that doesn't do this can find themselves in a lot more problems than if they did"

I think you're continuing to conflate monitoring (the failure to do so may have unfortunate legal consequences) and secret monitoring (the doing so which may have unfortunate legal consequences). Or are you actually saying that companies may be legally obligated to secretly monitor users?

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

smoogen — Thu, 09 Feb 2012 21:21:13 +0000

Well in the how it should be, I could understand that this sort of eavesdropping should be (if secret) should be illegal.

In how it is, that is a completely different thing. With the various laws and requirements from courts, a company that doesn't do this can find themselves in a lot more problems than if they did. That makes it more likely that they will put them in place. Now many of the companies put it in a banner or an HR slip that you sign saying any and all usage of company property can and will be monitored.. but depending on the location in the world they don't have to.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

dlang — Thu, 09 Feb 2012 20:21:14 +0000

you are assuming that the interception was secret from the employees. It's very common for companies large enough to deploy this sort of thing to notify the employees that their use of the company network will be monitored. If an individual then assumes that they are bypassing this policy, that's the individual's mistake.

the advantage of having a CA like this over a private one is that for the private one you have to update the valid CA list on every piece of software that is used in the company. Especially if you use mobile devices, this is a lot of work. I can see why this would have been an attractive option, while at the same time I think it's the wrong thing to do.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

gmaxwell — Thu, 09 Feb 2012 20:16:41 +0000

Network security. Many malware programs don't check certificate authenticity but will not be using the certs you plugged into a box even if you owned it.

Er. No. If they aren't checking they won't care if you're MITMing them. When an employer would provide a special certificate to be loaded for monitoring purposes it is not to actually permit the MITMing itself: you can be the man in the middle simply by virtue of being in the network path... It's to prevent the detection of/warning about the MITMing.

If you instead argued that the malware would detect the MITM because it would not use the installed roots and then not communicate (victory‽), I would counter that it would equally detect it due to a pinned cert.

Personel monitoring. People increasingly bring their own cell phones, laptops and then use the company network.

Like the above, not using a publicly trusted certificate does not inhibit the interception, it inhibits interception in secret (when client software is authenticating the certificate chain).

Certainly every browser user has the expectation that their SSL protected sessions are free of interception from their browser to the far end. As such I would expect and hope that undisclosed secret surveillance would be found to be unlawful, since the expectation of privacy has weighed heavily in every decision related to privacy in the workplace. ... especially because an easy alternative (a private certificate) is readily available and perfectly acceptable when the surveillance is not done in secret.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

smoogen — Thu, 09 Feb 2012 19:39:51 +0000

From my understanding, most of these certificates are placed in networks for 2 reasons:

1) Network security. Many malware programs don't check certificate authenticity but will not be using the certs you plugged into a box even if you owned it. The network security boxes then use those * certs to see what is going on anyway.

2) Personel monitoring. People increasingly bring their own cell phones, laptops and then use the company network. Case law has gone that anything posted there is the responsibility of the company to police.

Whether or not these make "common sense" or not, too many juries have ruled that companies are responsible for that and there is no safe harbour. So basically it becomes a rule that if you are large enough to sue, you better watch everything you can :/

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

rriggs — Thu, 09 Feb 2012 19:18:11 +0000

The thing is, you don't need a trusted cert for MITM attacks from within a private network if you have control of the computers on the network. You just need to issue yourself a CA cert and place that certificate into the list of trusted certs on your computers. That's not hard to do with any OS.

All that's required at that point is to proxy all the HTTPS traffic originating from the network and generate SSL certs for the destinations on the fly. (Which is the same thing you do when you have a subordinated root.)

There certainly are companies snooping SSL traffic originating from their employees' desktops, laptops and smartphones this way. Some of them are required to by law. You don't need a subordinated root of a trusted CA to do it.

What they did is just plain wrong.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

josh — Thu, 09 Feb 2012 18:36:42 +0000

Certificates issued for internal sites don't cause the problem mentioned in this article, unless you have a certificate which can in turn sign other certificates. You almost certainly don't.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

JoeBuck — Thu, 09 Feb 2012 18:30:14 +0000

My company (and no doubt many others) uses a Trustwave certificate for its Exchange server and other internal sites, so not trusting Trustwave isn't really an option.

A possible alternative for authorities known to operate in this manner is to have a way of trusting the cert only within a particular domain, say *.mycompany.com.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

josh — Thu, 09 Feb 2012 17:54:08 +0000

I think you've missed the severity of this problem by thinking that browsers should notice this. The CA model allows a trust chain from a certificate to a CA root by way of intermediate certificates, as long as those intermediate certificates have the bits set that allow that use. Among other things, CAs use this so they can store their trusted root offline and only use it to sign an intermediate root which in turn signs user certificates. Some CAs also use it to have various subsidiaries which they trust to offer certificates; see the Comodo issue where one of their subsidiaries got broken into.

TrustWave issued an intermediate root certificate, which could then sign any arbitrary certificate in a way that browsers would trust, and gave that intermediate root certificate to a company to MITM its employees. Browsers had no way to notice or warn users about this, unless they used one of the browser extensions that compares SSL certificates with what other people see, or happened to notice that all their SSL certificates came from the same intermediate root of the same obscure CA.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

josh — Thu, 09 Feb 2012 17:47:03 +0000

Reading the comments in the bug, someone suggested a potentially viable solution: mark the TrustWave root as not allowing any intermediate CA roots. Given the standard practice of issuing one intermediate certificate from an offline CA root and never signing user certificates with the root, Mozilla would need to whitelist the one legitimate CA root, but that seems acceptable.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

jimparis — Thu, 09 Feb 2012 17:40:56 +0000

> Hopefully this leads to an immediate removal of TrustWave from browser trust roots.

It's not clear what that would accomplish. There are plenty of CAs out there that probably did the same thing, and it seems out of place to punish TrustWave for both proactively revoking these subordinate certificates, and for publicly admitting their existence. More useful might be to say "Every other CA must similarly revoke such certificates by Feb 15; we'll start looking, and if we find any violations after that point, your CA will be immediately removed from the browser trust root forever". But as you say, the fundamental model of CAs is flawed.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

jd — Thu, 09 Feb 2012 17:40:53 +0000

If it's a "common industry practice", then there are problems with the common industry for a start. However, since the subordinate root can't match the signature of all the certs it is being used to match against, it also tells me that users aren't educated on data security and that at least some browsers are not flagging suspicious certs correctly.

(If the browsers were all flagging these certs and it was as common as is claimed, it would have been widely known before now. So one of those is not the case.)

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

josh — Thu, 09 Feb 2012 17:37:24 +0000

To clarify: it does seem unfortunate to apply this policy to a CA which came forward, admitted the problem, and revoked the certificate in question. However, given the *huge* amount of trust placed in CAs, and that the issuance of this certificate blatantly violates any and all sensible policies for certificate authorities, I don't see how Mozilla can do otherwise.

At a minimum, after clarifying their CA policy with an appropriate amount of "no really"s, CAs need re-validation against the new policy.

Trustwave admits issuing man-in-the-middle digital certificate (ComputerWorld)

josh — Thu, 09 Feb 2012 17:27:37 +0000

And yet again we have an example of the fundamental security model of certificate authorities: they protect you from anyone whose money they won't take.

Hopefully this leads to an immediate removal of TrustWave from browser trust roots.