New etcd, appc, and Rocket releases from CoreOS

Posted Feb 8, 2015 14:58 UTC (Sun) by kleptog (subscriber, #1183)
Parent article: New etcd, appc, and Rocket releases from CoreOS

I was curious about etcd/fleetd but was put off by the fact that internal security is practically non-existent. AFAICS any container using etcd can read/write the config of any other container using etcd. That's not really acceptable in some situations.

Has this changed recently?

New etcd, appc, and Rocket releases from CoreOS

Posted Feb 8, 2015 18:54 UTC (Sun) by raven667 (subscriber, #5198) [Link]

I doubt it but I don't think this kind of security is in-scope because containers in general are not currently suitable for multi-tenancy, their usefulness right now and with Docker or Rocket in particular, is in software lifecycle management, making it easy to install and uninstall cleanly on your machines. So if I were using this in production I'd create separate clusters for Internet facing apps and Intranet facing apps or wherever you have a hard security boundary based on the data being processed (PCI or HIPPA or trade secrets or whatever).

If someone breaks into your app server which is handling sensitive data then they probably have access to your sensitive data and etcd security is the least of your problems.

New etcd, appc, and Rocket releases from CoreOS

Posted Feb 8, 2015 21:15 UTC (Sun) by jberkus (guest, #55561) [Link] (5 responses)

Well, that depends on what you mean by "security".

I asked about security during the session. I was told that, while there's not password and user security for etcd, they have implemented full SSL security, including client machine certificates. So you can make sure that only machines you authorize can connect in the first place, and connections are encrypted.

If you think about it, this makes a lot of sense for etcd, since for systems using etcd, passwords are part of the etcd payload, so if you required user/password information to connect to etcd, you'd have a catch-22. etcd isn't a general-purpose key-value store; it's specifically meant for system configuration information.

I didn't report on this in the article above because I wasn't aware it was a change from 0.4. Is it?

New etcd, appc, and Rocket releases from CoreOS

Posted Feb 8, 2015 23:41 UTC (Sun) by dlang (guest, #313) [Link] (4 responses)

That's a good first step, but the parent poster was talking about a problem one layer higher. If you use etcd to manage your datacenter, can someone who breaks into a webserver use the connectivity to etcd to create an account on your database server for them to use?

If any machine can submit changes that affect other machines, that's a problem. If there is something in place to let a machine only change it's own data (with some sort of permission system to allow more, or only the master box can allow more), then it would not be a problem. Personally, I think a mode like this should be the default, but even if the default is wide open but can be locked down, it's acceptable.

If you can't lock it down, you would need to have a separate etcd setup/farm/cluster/whatever for each group of servers that you consider different from a security point of view (which could easily mean that all machines on the same network can be part of the same etcd setup in some networks)

New etcd, appc, and Rocket releases from CoreOS

Posted Feb 9, 2015 0:20 UTC (Mon) by jberkus (guest, #55561) [Link] (3 responses)

As far as I know, etcd has no permissions system. Hopefully one of the actual contributors will speak up here.

New etcd, appc, and Rocket releases from CoreOS

Posted Feb 9, 2015 8:18 UTC (Mon) by kleptog (subscriber, #1183) [Link]

dlang got it right. I have a real need for something like etcd but the lack of a security system means it won't fly in my situation. Hence I continue with my hand-made hand-managed kludge, which does work.

I've thought about adding it myself to etcd, but ISTM that if even basic permissions are not there yet then it looks like the developers don't consider security important. Which means I'd need to audit the entire codebase, as well as any new release when I wanted to upgrade. My kludge isn't that bad yet.

Which is unfortunate. The scene with containers (Docker, Rocket) is moving fast now and has the potential to make things a lot easier for everyone, yet no-one appears to consider security even slightly interesting. (See also Docker not verifying downloaded images).

With etcd, you wouldn't even have to go far. Suppose you could limit all writes to a handful of IP addresses you've addressed 99% of the problem. What remains is "is it a problem that you can read the configuration of other machines" which isn't a problem in many situations. Even now you could build something on etcd with encryption/signing to turn a compromise into merely a DOS.

Security isn't hard or complicated, you just have to consider it when building things.

Actually, the thought just occurred to me that you could layer The Update Framework on top of etcd. They've looked hard that the problem of secure updates, delegation, recovering from compromise, etc. Interesting weekend project.

New etcd, appc, and Rocket releases from CoreOS

Posted Feb 12, 2015 3:35 UTC (Thu) by krakensden (subscriber, #72039) [Link] (1 responses)

Does Consul (one of the closest competitors)? They talk a good game about symmetric keys, but it's hard to tell.

New etcd, appc, and Rocket releases from CoreOS

Posted Feb 12, 2015 7:52 UTC (Thu) by kleptog (subscriber, #1183) [Link]

Thanks for the tip. Looking at the documentation they don't talk about it, but because they use plain HTTP you could stick a proxy in front of it (say nginx) to restrict PUT/DELETE to particular IPs. They support a recurse option, so you'd have to find a way to restrict that.

etcd also uses HTTP, but the certificates part might make it hard to MITM it like that.

Still, new possibilities...