Another daemon for managing control groups [LWN.net]

Another daemon for managing control groups

Posted Dec 10, 2013 17:13 UTC (Tue) by khim (subscriber, #9252)
In reply to: Another daemon for managing control groups by Cyberax
Parent article: Another daemon for managing control groups

Really? What's so complicated in the interface which is supposed to just start the program? It just needs to check credentials and do that. It always start applications in pre-determined environent with known starting conditions.

Compare with today's approach where bazillion parts of kernel must know about suid bit (euid vs uid), many libraries need to know about suid bit (euid vs uid), glibc must specifically handle startup of setuid binaries (and there were many exploits around this process), binaries often need special handling if they are supposed to ever run as suid binaries. Sorry, but argument is nor convincing.

Note that even today when suid bit is actually available many programs are not using it and use cetralized-privileged-daemon scheme instead (things like apache, ftp, mysql and other countless daemons). Strange, isn't it?

Sorry, but setuid bit is obviously a mistake. It's not easy to replace setuid bit with a DBUS interface today and perhaps it's not even worth trying (transition pain can easily outweight and potential gain), but the design itself is obviously too complex and too fragile. That's not even worth discussing.

to post comments

Another daemon for managing control groups

Posted Dec 10, 2013 18:10 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (11 responses)

>Really? What's so complicated in the interface which is supposed to just start the program? It just needs to check credentials and do that. It always start applications in pre-determined environent with known starting conditions.

Because it will have ALL the faults of suid and lots of additional faults of a half-baked userspace implementation. For example, think about signals (especially RT signals and SIGSTOP/SIGKILL). I can kill my SUID program using a straightforward "kill" utility, how would you do this with SuidD?

I'm actually speaking from experience - we have such a daemon in our system. It's simply not possible to replicate all the kernel-level functionality.

SystemD is repeating ALL the problems of this approach. For example, they have to cobble something together to handle delegation to containers while simple bind-mount is enough right now to nest cgroups.

Another daemon for managing control groups

Posted Dec 10, 2013 21:46 UTC (Tue) by khim (subscriber, #9252) [Link] (10 responses)

Because it will have ALL the faults of suid and lots of additional faults of a half-baked userspace implementation.

Really? Which ones?

For example, think about signals (especially RT signals and SIGSTOP/SIGKILL).

What about signals?

I can kill my SUID program using a straightforward "kill" utility

Yup. And it is a problem security-wise.

how would you do this with SuidD?

Most likely answer: you would not be able to do that. You will probably have some high-level knobs but you will not be able to just send random signals to random provileged programs. And this is “good thing”™.

I'm actually speaking from experience - we have such a daemon in our system. It's simply not possible to replicate all the kernel-level functionality.

Of course not! It'll be pointless excercise to just shuffle functionality around. It's the other way around: suid is a problem because it gives you huge amount of rope to tie itself. SuiD will give you much, much smaller amount of rope. Yes, this will also mean that some brain-dead designs will become impossible, but this will just mean that you will need to spend few more time thinking about design of your system upfront. What real-world task are you trying to solve with signals? Why do you think it can only be solved by giving the rights to affect priveleged process on your system from some random shell script?

SystemD is repeating ALL the problems of this approach.

Wow. Thanks for bringing that to my attention. Seriously, no sarcasm. This cgroups ≈ suid analogy really helps to show just why it's bad idea to give access to just some random user to the capabilities of cgroups… but it still does not explain why only one daemon can ever manipulate cgroups. Ok, It needs to be privileged daemon, but it's still not entirely clear to me just why it must be PID 1.

For example, they have to cobble something together to handle delegation to containers while simple bind-mount is enough right now to nest cgroups.

Well, it was always good idea to have daemon which does that thus I'm not sure why you are just now trying “to cobble something together”. The problematic fact is that all these solutions must be tied somehow to systemd, they can not just exist as yet-another-daemon on the side, but this is not systemd's fault, AFAICS it was imposed by kernel side changes.

Another daemon for managing control groups

Posted Dec 10, 2013 22:39 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (7 responses)

>Most likely answer: you would not be able to do that. You will probably have some high-level knobs but you will not be able to just send random signals to random provileged programs. And this is “good thing”™.

And here's the problem - it's "The my way or the highway".

Let's discuss a very simple SUID program - good old 'ping' utility. A user should be able to watch live its output, so some kind of shim utility should be used to transfer standard FDs to the DBUS service. This shim must also be running all the time while the DBUS ping service is running.

So far so good. But now I want to stop the service, so I press Ctrl-C. And nothing happens, unless the shim captures this signal and somehow communicates it to the DBUS service (oh, and don't forget to authenticate the transmission).

So far so good, until I press Ctrl-Z. Whoops. SIGSTOP can't be captured.

And that's without going into the gory details of controlling terminals, ptys and realtime signals (can you say 'priority inversion'?). It doesn't matter that YOU don't like signals - they are de-fact used in the world out there.

But let's go on. Suppose that we have a browser running in a sandbox. Should it be able to access DBUS? Likely. But I definitely don't want it to access the SUID-runner service, while my beloved Tilda should be able to start whatever processes I want. Can you tell me how DBUS services are secured? How can I audit this security? Can I write an AppArmor policy to restrict '/usr/bin/firefox' access to '*cgroup*'?

Oh, and we have this nice Criu project - but it won't be able to checkpoint the DBUS-based service (it can't checkpoint only one end of a Unix socket).

And we can leave out minor details like confusing 'ps' output.

In the end, the DBUS-based solution is going to be an inferior and unreliable construct. And that's exactly what is happening with SystemD and cgroups right now. They are building an inferior wrapper on top of a kernel interface, that's in itself WORSE than the status quo.

>Wow. Thanks for bringing that to my attention. Seriously, no sarcasm. This cgroups ≈ suid analogy really helps to show just why it's bad idea to give access to just some random user to the capabilities of cgroups…
Yes, probably there are several tight spots in the cgroups API that might give users too much capabilities to harm the system. But so does /sys, /proc and namespaces - yet all of them are accessible to users.

Another daemon for managing control groups

Posted Dec 11, 2013 0:35 UTC (Wed) by khim (subscriber, #9252) [Link] (3 responses)

And that's without going into the gory details of controlling terminals, ptys and realtime signals (can you say 'priority inversion'?). It doesn't matter that YOU don't like signals - they are de-fact used in the world out there.

It's not about signals. It's about system design. Any time indirection goes from unprivileged process to privileged one it must be accounted and cotrolled. It's really hard to do with setuid approach and most programs don't bother to do that.

Let's discuss a very simple SUID program - good old 'ping' utility.

Sure, let's do that. Consider the fact that said utility plays with very low-level stuff and can easily hurt not just your system but also neigbhoring systems. Let's see if we can actually do that:
$ ping -f www.google.com
PING www.google.com (74.125.143.106) 56(84) bytes of data.
ping: cannot flood; minimal interval, allowed for user, is 200ms

Wow! Lookie: there are a protection! But does it actually work? Of course not: you can still run 1000 ping's in parallel and this will have basically the same effect.

In most cases what you need it something similar to tcptraceroute -f 30 -q 10 (which works without any special permissions), anyway.

So far so good. But now I want to stop the service, so I press Ctrl-C. And nothing happens, unless the shim captures this signal and somehow communicates it to the DBUS service (oh, and don't forget to authenticate the transmission).

So far so good, until I press Ctrl-Z. Whoops. SIGSTOP can't be captured.

Which is good because it's NOT good idea to do something to highly privileged process behind it's back. Actual priveleged ping process may notice that shim is no longer responding and will probably stop doing it's work. That's fine, don't see anything wrong with that.

Basically you are explaining why current [broken] interface is hard to replicate with SuiD deamon. That's fine, I agree with you: it's really hard to replace it with anything sane and perhaps it's not ever a good idea to try to do that right now. It still does not mean that it was good idea to build it in this form initially.

But let's go on. Suppose that we have a browser running in a sandbox. Should it be able to access DBUS? Likely. But I definitely don't want it to access the SUID-runner service, while my beloved Tilda should be able to start whatever processes I want.

How is it different from /proc or /sys access?

Yes, probably there are several tight spots in the cgroups API that might give users too much capabilities to harm the system. But so does /sys, /proc and namespaces - yet all of them are accessible to users.

And that's why we must assume that any process started under any user yet with full access to all syscalls and /proc and /sys it having root access more or less automatically. It's basically impossible to make Linux kernel secure because it's attack surface is so wast.

Looks like people are really starting to think about it, but it's hard to change everything at once thus they are starting from most recent piece of the puzzle (which can be changed without affecting too many users yet). I'm just not sure what they are planning to do after that: sure, they will secure one tiny pice of the whole, but how exactly it'll help if everything else will remain in the same hodge-podge-with-bazillion-security-holes state?

Another daemon for managing control groups

Posted Dec 11, 2013 0:51 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

>It's not about signals. It's about system design. Any time indirection goes from unprivileged process to privileged one it must be accounted and cotrolled.
Welcome to Linux audit framework (that nobody uses, but it's there).

>Sure, let's do that. Consider the fact that said utility plays with very low-level stuff and can easily hurt not just your system but also neigbhoring systems.
Irrelevant. This particular warning is obsolete, anyway. I can just as well flood the network with UDP datagrams.

>Which is good because it's NOT good idea to do something to highly privileged process behind it's back.
Nope. It's a good idea, because SUID processes are specifically meant to interact with users. And SIGSTOP is one of the well-known ways to interact.

>Actual priveleged ping process may notice that shim is no longer responding and will probably stop doing it's work. That's fine, don't see anything wrong with that.
So there should be a heartbeat service? What about power consumption (all those spurious wakeups)?

You're digging hole even deeper.

>> But let's go on. Suppose that we have a browser running in a sandbox. Should it be able to access DBUS? Likely. But I definitely don't want it to access the SUID-runner service, while my beloved Tilda should be able to start whatever processes I want.
>How is it different from /proc or /sys access?
That's it - it's not different at all. Except that I have easy to use tools to restrict access to /sys and /proc - AppArmor or SELinux (for masochists). I'm not aware of similar infrastructure for DBUS.

>And that's why we must assume that any process started under any user yet with full access to all syscalls and /proc and /sys it having root access more or less automatically. It's basically impossible to make Linux kernel secure because it's attack surface is so wast.
Container people managed to fix this. It's possible to start a namespaced container with its own view of /proc and /sys with full root access in it and it will be reasonably secure.

And puzzle comparison is apt - for many years full container support was known as 'containers puzzle' (just search LWN). Many people diligently chipped away all the pieces to make full isolation possible. And it's finally there.

Except now cgroups developers say: "It's too complicated for us, we'll just throw in the towel and make it impossible even if it works right now for many users. For their own good."

Another daemon for managing control groups

Posted Dec 11, 2013 10:33 UTC (Wed) by zdzichu (subscriber, #17118) [Link] (1 responses)

It's built-in in D-Bus. See http://dbus.freedesktop.org/doc/dbus-daemon.1.html (search for policy) or content of /etc/dbus-1/ directory.

BTW. the proper spelling is "systemd" (no arbitrary uppercase letters).

Another daemon for managing control groups

Posted Dec 11, 2013 19:38 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Nope. Can't use this policy to limit access to certain _processes_.

So we have two "solutions" already: polkit and DBUS policies. Which one is it?

Another daemon for managing control groups

Posted Dec 11, 2013 3:41 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

> Can I write an AppArmor policy to restrict '/usr/bin/firefox' access to '*cgroup*'?

There's polkit (which is used to restrict access to APIs such as sleep/hibernate/shutdown) which can be used. I don't know where it acts though (whether at the dbus-daemon level or the receiver making *another* call out to polkit asking for permission.

Another daemon for managing control groups

Posted Dec 14, 2013 12:48 UTC (Sat) by kleptog (subscriber, #1183) [Link] (1 responses)

Is it just me, or is this example flawed:

Let's discuss a very simple SUID program - good old 'ping' utility. A user should be able to watch live its output, so some kind of shim utility should be used to transfer standard FDs to the DBUS service. This shim must also be running all the time while the DBUS ping service is running.
So far so good. But now I want to stop the service, so I press Ctrl-C. And nothing happens, unless the shim captures this signal and somehow communicates it to the DBUS service (oh, and don't forget to authenticate the transmission).

Pressing Ctrl-C is different from sending a SIGINT. Namely, when you press Ctrl-C, the kernel sends a SIGINT to anything using that terminal. I imagine the shim would pass through all necessary file descriptors and hence CTRL-C will work fine.

Would it be weird if you were allowed to Ctrl-C a process, but not be allowed to send it a signal from another terminal?

(Hmm, ping drops back to the normal user after opening the socket, does that mean another process could ptrace it and get access to the socket that way? ptrace block it now, but it is something to consider)

Another daemon for managing control groups

Posted Dec 14, 2013 18:47 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

The shim is using the controlling terminal so it'll get a signal. Not the privileged binary. I checked.

Anyway, you'll still have the problem with SIGSTOP.

Another daemon for managing control groups

Posted Dec 11, 2013 3:36 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (1 responses)

> but it's still not entirely clear to me just why it must be PID 1.

It doesn't *have* to be, just that with systemd, PID 1 is the simplest place to put it since it already dances around with cgroups pretty heavily and IPC to another service to manage cgroups makes that service a Special Snowflake that can't be set up using cgroups (since it would have to answer questions about how to start it before it starts).

Another daemon for managing control groups

Posted Dec 11, 2013 16:15 UTC (Wed) by khim (subscriber, #9252) [Link]

Well, that's the problem: PID 1 needs cgroups to manage services and container manager (presumably well-tested and quite privileged) needs them too. Why could not they both use cgroups? Why one must be client of the other one? That requirement was never properly explained.