LWN: Comments on "The trouble with CAP_SYS_RAWIO"

The trouble with CAP_SYS_RAWIO

Duncan — 2013-03-31T07:12:07+00:00

> the problem is that these 'iffy' scsi commands end up being things like the ability to burn CDs, not exactly easy things to do without.

Actually, that's rather less of a problem now, and trending less so, than it was a few years ago when /the/ major form of removable media was optical, CD/DVD. Now days, the sub-GB size of a CD looks positively diminutive, and even the near-5-GB size of a standard DVD looks small, compared to the ubiquitous USB thumbdrive of say 8+GB. A Bluray's 25 gigs is a bit better priced media-only (US$6 individual, just over $1/ea in 25-packs, pricewatch.com), but while the stick's a bit more expensive (USB flash: 32G=US$15, 16G=$10) as it ships with its own housing and read/write hardware, it's also CONSIDERABLY less fragile and generally more easily handled, AND direct-block-device read-writable (well, as seen by the OS...).

Additionally, with current inet and smartphone penetrations, people that a few years ago might have used dedicated removable media (either USB sticks or CD/DVD) these days more often either use the inet directly (streaming what might have been on CD a few years ago, or pastebinning it to a friend if not attaching it to an email), or if they do play local media, say in the car, it's from a jacked-in phone more often than a CD.

Unfortunately when I upgraded machines last year I didn't think of that, and bought a blu-ray burner for it. I really haven't used it... Fortunately, it's a USB-based one and wasn't /that/ expensive, so it's usable on my netbook as well should I decide to and not taking any power when it's unplugged (see kernel 3.9's new ZPODD, only I've had that in the form of an unplugged USB-based bluray for a few months now), and being USB, as long as I don't mistreat it it should stay usable for years, so I suppose I'll get some use out of it, over time. But I'd have been better off simply not buying it at all.

So it's considerably easier to do without optical burning than it was even just a few years ago, to the point where many people would miss it about as much as they do their 1.44 MB floppy...

The trouble with CAP_SYS_RAWIO

mkerrisk — 2013-03-25T13:16:18+00:00

> but why not put it in the limelight of the first paragraph, where it rightly belongs?

Reviewing the article, it's a smart editorial suggestion and I agree with you; and it would have made for a punchier start. Next time...

The trouble with CAP_SYS_RAWIO

clemenstimpler — 2013-03-22T23:32:26+00:00

It takes a bit, until curious, but uninformed readers learn what this article is about. "CAP_COMPROMISE_KERNEL (formerly known as CAP_SECURE_FIRMWARE) is a new capability designed for use in conjunction with UEFI secure boot, which is a mechanism to ensure that the kernel is booted from an on-disk representation that has not been modified" would have made a great first sentence for this article. It's also nice at the end of the second paragraph, but why not put it in the limelight of the first paragraph, where it rightly belongs? Just saying... ;)

The trouble with CAP_SYS_RAWIO

dlang — 2013-03-22T07:12:41+00:00

the problem is that these 'iffy' scsi commands end up being things like the ability to burn CDs, not exactly easy things to do without.

Or in enterprise settings, commands to be able to manipulate tape changers

The trouble with CAP_SYS_RAWIO

kevinm — 2013-03-22T04:36:42+00:00

It still sounds to me like the simple solution is "remove CAP_SYS_RAWIO from the initial capability set on secure-booted kernels". So you'll lose the ability to perform some iffy SCSI commands - well, you signed up for some bondage and discipline when you asked for a locked-down, secure-booted kernel, didn't you?

*BSD "securelevel"

renox — 2013-03-20T09:13:20+00:00

Is capsicum actually used in FreeBSD?
AFAIK it's only useful iff programs are ported to use it..

*BSD "securelevel"

wahern — 2013-03-19T22:33:58+00:00

There are enough issues w/ BSD securelevel, too, that it doesn't receive much interest these days.

For example, the immutable files protection can be bypassed by mounting over the directory. It doesn't allow you to change the original file, but allows you to fool other applications at runtime and is thus of little use for, e.g., preventing root kit installation once you've already attained root.

AFAIK nobody has bothered to fix it on systems where it was an issue (NetBSD was immune to this particular attack). The fundamental issue is that even this course-grained capabilities system gives a false of security. Invariably someone will forget about some corner case, or some new feature is added which allows circumvention of the whole pile of policies.

Fine-grained capabilities systems (both system-level and process-level) are just too brittle, including the policies, the mechanisms, and the actual implementations.

Unix systems have only just recently reached a decent level of correctness and reliability with basic file permissions. Anybody who relies on more sophisticated schemes (or allows them in their kernel) is just begging to be rooted.

The trouble with CAP_SYS_RAWIO

wahern — 2013-03-19T01:57:11+00:00

"So the end result is opening up more attack surface to the most vulnerable part of the system, and soon on all distros you will have no choice but to be exposed to it."

Unfortunately, "less code, simpler code" is not one of the competing security paradigms in Linux Land.

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-17T17:02:35+00:00

Any application that drops all privileges other than CAP_SYS_RAWIO before attempting the open?

The trouble with CAP_SYS_RAWIO

mrjk — 2013-03-17T15:46:31+00:00

With the suggested change there would be no program that used to work that would not work now that I can see. Every single current program that worked with dropping privileges after an open would still work the exact same way with caching the new capability at open time and using the cached value on those opened channels.

Can you give an example that would now break -- that wouldn't have broken already?

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-14T20:03:53+00:00

A program used to work. After a change, that program no longer works. A non-working program is broken. Breaking that program doesn't add any real additional security in the common (ie, non-Secure Boot) case, and so is undesirable.

The trouble with CAP_SYS_RAWIO

WolfWings — 2013-03-14T19:52:30+00:00

Actually I don't believe the suggestion change (move the check for write-modes to the open()) breaks userspace. Not the 'silent downgrade to read-only' which is the entire problem right now and identical to the 'check on write()' problem.

Right now you don't appear to be able to drop-all-caps then open /dev/msr, you need to open it first then drop privs as there already is a check to block so much as reads unless you have the RAWIO cap.

So how does the "read = RAWIO, write = RAWIO && COMPROMISE" check on open() instead of on write() break userspace? Programs would be refused access to /dev/msr and complain about it, same as before, and their existing 'Check your caps!' error messages would still apply.

There's a difference between 'breaking' userspace in a way that existing apps error messages don't apply and may not even have error-handling paths for the new issues, and simply enforcing stronger checks in a way compatible with existing error handling.

*BSD "securelevel"

ewen — 2013-03-14T18:35:19+00:00

Which is an important point: if you're going to implement something like this, it needs to be sufficiently encompassing (in the single switch) that it doesn't allow a user (or attacker) to simply undo it by using an alternative mechanism to manipulate the value. The *BSD "securelevel" handles that by having a single switch that cuts off a set of things, including raw memory manipulation and module loading.

It sounds like the "compromise the kernel" flag is also aimed to cut off a more encompassing set of things, so hopefully it'd be less easy to do an end-run around the protection. (And there'd be more incentive to add "you can't do that either" into the set of things turned off as other ways to manipulate it are discovered: Firewire device DMA being one that comes to mind.)

Ewen

*BSD "securelevel"

ThinkRob — 2013-03-14T16:39:17+00:00

At the risk of going slightly off-topic (though not much, given the matter at hand), what do you think of FreeBSD's approach to kernel security? I'd be willing to wager you're a fan of the fact that (it seems like) they're a bit more up-front with their security hole disclosure, but what about things like the "secure level" model, jails, capsicum, and other security-oriented features they've introduced -- do you think they're on the right track?

I know the full answer to this is probably not terribly succinct, but I'm just curious to hear what you think, since kernel security is obviously something you're quite passionate about. :)

The trouble with CAP_SYS_RAWIO

paulj — 2013-03-14T15:30:37+00:00

And looking into this, cap_user_header_t actually has a version field! So this problem may well be fixable without hacks or breakage, if the kernel developers start to consider that the semantic meaning of capabilities, as exposed to user-space, is versioned.

The trouble with CAP_SYS_RAWIO

paulj — 2013-03-14T12:55:20+00:00

Oh, in other fields of computer science, changing semantics of protocols in incompatible ways is often handled through versioning.

I.e. the capability calls need a version flag, perhaps?

The trouble with CAP_SYS_RAWIO

paulj — 2013-03-14T12:35:29+00:00

It seems pretty clear from this article that the root of this problem is that the semantic meaning of certain capabilities either was unclear at the start, or was allowed to degenerate from a precise meaning into a mess.

*BSD "securelevel"

spender — 2013-03-14T11:40:34+00:00

Linux already attempts this a bit inconsistently. See /proc/sys/kernel/modules_disabled, which does not allow module loading to be re-enabled once disabled. Yet you can just run: https://grsecurity.net/~spender/msr32.c (before it required no capabilities, now it requires CAP_SYS_RAWIO, but neither matter) and re-enable the supposed securelevel-like value. There is no CVE for this (only for the adding of CAP_SYS_RAWIO) and no patch for the removal of modules_disabled on x86. There's no coherency to upstream security.

-Brad

The trouble with CAP_SYS_RAWIO

spender — 2013-03-14T11:33:09+00:00

Key phrase being "this specific issue". Several vulnerabilities have already been introduced by the addition of unprivileged user namespaces. It speaks to the trustworthiness of the code that these were found through casual inspection of a few lines of code and a very dumb fuzzer (trinity) -- it has not been exposed to serious security auditing. The author of the above exploit said it was the easiest he has ever written.

You should also know that the existing kernel exploit payloads for granting root privilege also break out of user namespaces without modification.

So the end result is opening up more attack surface to the most vulnerable part of the system, and soon on all distros you will have no choice but to be exposed to it. It's just broken security design.

-Brad

*BSD "securelevel"

ewen — 2013-03-14T10:03:12+00:00

Reading through this article reminded me of *BSDs "securelevel", which is a one-way ratchet change (ie having changed it, you can't change it back to a less secure level except by rebooting). It controls various "compromise the kernel" like things. The exact set of things it controls is probably not an ideal match, but the idea of a sysctl value which can only ever be changed to "be at least as restrictive of insecure things you can do as now" seems like a fairly good fit. And it would be completely orthogonal to the Linux capabilities, which seems helpful. (As well as being "system wide" which seems desirable in this case -- if you've booted via secure UEFI you probably don't want to end up in a situation where some processes can compromise the kernel and others cannot....)

Ewen

The trouble with CAP_SYS_RAWIO

kees — 2013-03-14T06:51:28+00:00

Yes, and this specific issue has already been fixed:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linu...

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-14T05:04:55+00:00

CAP_SYS_RAWIO is not equivalent to CAP_SYS_ADMIN. That's why they're not defined to the same value. It does not imply the ability to arbitrarily alter the system. That's *the entire point* of capabilities.

The trouble with CAP_SYS_RAWIO

heijo — 2013-03-14T04:50:23+00:00

The problem is that both CAP_SYS_RAWIO, CAP_SYS_ADMIN and possibly others used to be equivalent and imply the ability to arbitrarily alter the system.

Redefining those to no longer being able to do so is idiotic and breaks compatibility.

The trouble with CAP_SYS_RAWIO

shlevy — 2013-03-14T03:12:17+00:00

Yikes!!! I had to disable fs.protected_hardlinks, but I can confirm this exploit works... Has this been reported to the appropriate channels?

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-13T22:48:23+00:00

You've still broken userspace. It's still undesirable and it's still a fundamental problem with the capabilities interface. Similar changes in other places in the Fedora kernel have broken various bits of userspace in irritating ways.

The trouble with CAP_SYS_RAWIO

smurf — 2013-03-13T22:40:53+00:00

It fixes the "open the device and then drop privileges" usage.

It does not fix the "drop privileges and then open the device" case. I know that. But if userspace runs afoul of this problem, the fix is a simple re-ordering of two lines of code. Or retaining an additional capability.

Is there any real-world program that would run aground at this change, or is this an academic exercise?

Personally I never regarded CAP_SYS_RAWIO (or _ADMIN) as written in stone. It's a sufficiently broad catch-all category that a reasonable programmer should expect that requiring a different, or additional, capability for a task that used to work with only this one right might be in the cards.

The trouble with CAP_SYS_RAWIO

spender — 2013-03-13T22:10:38+00:00

Get busy fixing this trivial local root vulnerability you introduced in 3.8 first:
http://stealth.openwall.net/xSports/clown-newuser.c

-Brad

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-13T21:49:28+00:00

Imagine a /dev/halt_catch_fire that requires CAP_SYS_RAWIO. Your application runs as root but drops all capabilities, so you never bother making sure that it doesn't accidentally touch /dev/halt_catch-fire. Later, someone decides to add a more fine-grained capability and now either CAP_SYS_RAWIO *or* CAP_SYS_HALT_CATCH_FIRE is sufficient. Your application was previously incapable of setting the machine on fire, but now it can.

The trouble with CAP_SYS_RAWIO

ebiederm — 2013-03-13T21:39:10+00:00

Just for fun I will suggest changing capable from:

bool capable(int cap)
{
return ns_capable(&init_user_ns, cap);
}

to:

bool capable(int cap)
{
if (we_dont_trust_root)
return false;
return ns_capable(&init_user_ns, cap);
}

Which is equivalent to running userspace outside the initial user namespace, and trivially gives you and environment that has been audited to work for an untrusted root.

Just a few more things won't work that way but I would not mind a little help flushing out the things that we can trust less than fully privileged users with doing.

As for msrs. Make no mistake someone will eventually implement rdmsr(HALT_AND_CATCH_FIRE). So I can't believe even reading msrs is safe.

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-13T21:38:34+00:00

The current state of things: an application that drops all capabilities other than CAP_SYS_RAWIO can read and write MSRs. The new state of things: an application that drops all capabilities other than CAP_SYS_RAWIO can no longer write MSRs. So, how does that fix anything?

The trouble with CAP_SYS_RAWIO

kugel — 2013-03-13T21:36:36+00:00

Can the cap dropping be changed to happen in a way such that only those capabilities are dropped that existed when the program was compiled? That said, I know nothing about capability APIs or internals.

The trouble with CAP_SYS_RAWIO

khim — 2013-03-13T21:25:17+00:00

Application which drops CAP_SYS_COMPROMISE_KERNEL will work just fine because both checks happen in open(2) syscall. It'll break application which opens file for reading and writing but then only issues read commands. This can be fixed by changing logic: if read/write open(2) request is attempted without CAP_SYS_COMPROMISE_KERNEL then it's silently translated to read-only open(2) request. Of course application which will try to write to said file will see EBADF which may crash it, but I'm not sure what can save such an application.

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-13T20:20:21+00:00

Applications that drop CAP_SYS_COMPROMISE_KERNEL are still unable to write. How does this help?

The trouble with CAP_SYS_RAWIO

smurf — 2013-03-13T20:11:57+00:00

No, the check should be

if (reading && !CAP_SYS_RAWIO) return -EPERM;
if (writing && !CAP_SYS_COMPROMISE_KERNEL) return -EPERM;

when the device is opened, and anything subsequent checking the file's flags.

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-13T19:59:53+00:00

Which would still result in any application that drops CAP_SYS_COMPROMISE_KERNEL before doing anything else being unable to write.

The trouble with CAP_SYS_RAWIO

cesarb — 2013-03-13T19:49:12+00:00

No, the check would be "if (!capable(CAP_SYS_RAWIO)) return -EPERM;" followed by stashing the value of capable(CAP_SYS_COMPROMISE_KERNEL) somewhere. When writing, check the stashed value instead of capable(CAP_SYS_COMPROMISE_KERNEL) directly.

The trouble with CAP_SYS_RAWIO

mjg59 — 2013-03-13T17:47:06+00:00

The check would be if(!capable(CAP_SYS_RAWIO) || !capable(CAP_SYS_COMPROMISE_KERNEL)) return -EPERM, so anything that drops all capabilities other than CAP_SYS_RAWIO would still fail.

The trouble with CAP_SYS_RAWIO

cesarb — 2013-03-13T17:21:23+00:00

For the particular scenario mentioned: couldn't CAP_COMPROMISE_KERNEL be checked at open time (together with CAP_SYS_RAWIO) and set a flag in the kernel open file struct? That way, dropping capabilities after open would not cause any problems, and the application would keep working as it used to.