|
|
Subscribe / Log in / New account

Using systemd for more secure services in Fedora

By Jake Edge
December 21, 2016

The AF_PACKET local privilege escalation (also known as CVE-2016-8655) has been fixed by most distributions at this point; stable kernels addressing the problem were released on December 10. But, as a discussion on the fedora-devel mailing list shows, systemd now provides options that could help mitigate CVE-2016-8655 and, more importantly, other vulnerabilities that remain undiscovered or have yet to be introduced. The genesis for the discussion was a blog post from Lennart Poettering about the RestrictAddressFamilies directive, but recent systemd versions have other sandboxing features that could be used to head off the next vulnerability.

Fedora project leader Matthew Miller noted the blog post and wondered if the RestrictAddressFamilies directive could be more widely applied in Fedora. That directive allows administrators to restrict access to the network address families a service can use. For example, most services do not require the raw packet access that AF_PACKET provides, so turning off access to that will harden those services to some extent. But Miller was also curious if there were other systemd security features that the distribution should be taking advantage of.

He suggested perhaps having a Rawhide flag day where the network address families were restricted by default using the directive (to, say, only AF_INET, AF_INET6, and AF_UNIX); that would allow enough time to find services that needed a less (or more) restrictive set of address families and override those defaults in their unit files. An alternative to changing the defaults might be to file bugs for each affected package, Miller said, though Tomasz Torcz thought those bugs should be filed with the upstream project, rather than in the Fedora bug tracker, to reduce divergence with upstream.

Ratcheting down the availability of unneeded address families, which systemd does using the kernel's seccomp facility, appeared to be a reasonable idea based on the responses of those participating in the thread. The need for a flag day was less clear. With his "upstream hat on", as a Libreswan developer, Paul Wouters asked about ways to support the feature without requiring systemd (since Libreswan runs on non-Linux systems too). While Poettering seems to have misinterpreted Wouters's initial message (so Wouters clarified it), Poettering did describe the mechanism to do so:

Of course, you can also set up seccomp filters yourself, in your daemon, in C code, by using libseccomp. It's great if you do, and that's totally possible, and can be functionality-wise entirely equivalent. The only difference is: systemd makes all of this trivially easy to use, by making this a single-line change in a unit file without involving C hacking.

There are other systemd features that might be used by services in Fedora, though; Poettering listed fourteen separate sandboxing directives that Fedora "should enable wherever we can". They are well documented, he said, though not well publicized at this point. Most are available in the systemd that ships with the currently maintained Fedora versions, though not all are.

Some of the directives he listed include:

  • ProtectSystem: This allows services to choose to mount some filesystems read-only for their processes. It defaults to "false"; setting it to "true" will create a new mount namespace and mount the /usr and /boot directories read-only in it. The "full" setting adds /etc to that list, while "strict" does it for the entire filesystem hierarchy except for /dev, /proc, and /sys (which can be individually protected in various ways using the PrivateDevices, ProtectKernelTunables, and ProtectControlGroups directives).
  • ProtectHome: This directive will set the /home, /root, and /run/user directories either as inaccessible and empty (if set to "true") or as read-only (if set to "read-only") for all processes in the unit. It defaults to "false".
  • ProtectKernelModules: If set to "true", kernel module loading will be disabled for the service. It removes the CAP_SYS_MODULE capability, installs a system call filter to block module-loading system calls, and makes /usr/lib/modules inaccessible. All of that does not stop automatic loading of kernel modules, though, which can be done system-wide using /proc/sys/kernel/modules_disabled.
  • PrivateTmp: This will cause systemd to create a new mount namespace for the unit with a private /tmp and /var/tmp that are not shared with any processes outside of the unit.
  • PrivateNetwork: If enabled, systemd sets up a private network namespace for the service with only the loopback device available to it. This effectively turns off all network access (except to Unix sockets available in the filesystem) for all of the unit's processes.
  • MemoryDenyWriteExecute: This will install a system call filter that removes the ability of the service and any of its children to create memory mappings that are both writable and executable. It intercepts attempts to use mmap() with both PROT_WRITE and PROT_EXEC, mprotect() with PROT_EXEC, and shmat() with SHM_EXEC. The idea is to restrict a program's ability to modify the code it runs, which can be exploited in various ways, but it is incompatible with programs that are meant to do that (e.g. for just-in-time compilation) so it can only be enabled for some services.
  • RestrictRealtime: This boolean directive can block any attempt by a process in the unit to enable realtime scheduling, such as SCHED_FIFO, SCHED_RR, and SCHED_DEADLINE.

There are others, of course. Another useful pointer that came out of the discussion was the systemd.directives man page that Poettering noted, which has entries for each systemd directive. Those entries are linked to the proper spot elsewhere in the systemd man pages to get the full description of the directive.

There was some discussion of trying to enable some of the sandboxing options by default, but Poettering thinks that ship has sailed:

If we'd globally say that all services now run with ProtectSystem=strict by default, then we'd break pretty much all services that want to write something anywhere, until they get updated with the right ReadWritePaths= statements... And I have the suspicion that this kind of churn would upset quite a few people... I mean, I am all for breaking eggs to make an omelette, but not maybe not break all eggs in the egg carton at once ;-)

But Japheth Cleaver thought that efforts to use systemd's sandboxing facilities would be better spent elsewhere: "I'd much rather that effort be put into good SELinux policy evangelization, documentation, and perhaps additional admin-controllable booleans." No one really objected to that idea, exactly, but the SELinux complexity problem reared its head. As Poettering put it:

Yeah, this is really what it boils down to: the goal with the systemd directives is to make things easy to grok and easy to change. I can probably explain to most Linux admins who have administered a current Fedora in 5min what ProtectSystem=strict and ReadWritePaths=/var/lib/myservice does, and why it's a good thing. And afterwards he can easily add this to his own services. With SELinux that's not that easy: the concepts are much more complex (at least in my opinion, but I am sure many will agree), and as the selinux policy is packaged centrally making a change is not trivially easy to do.

That said, SELinux and the systemd sandboxing directives are very different concepts. I don't think they are in competition really, and I am pretty sure everybody would benefit if both the SELinux policy and the systemd unit files would be improved.

Certainly providing better protections for system services of various sorts can only lead to more secure systems. Systemd has been adding many features to make it easier (and, it should be said, more understandable) for software projects, system administrators, distributions, and others to enable those protections in fairly straightforward way. In addition, doing it that way has the advantage of spreading the protections throughout the mainstream Linux distribution ecosystem. All of that added together makes it a project worth tackling.


Index entries for this article
SecurityDistribution security


to post comments

Using systemd for more secure services in Fedora

Posted Dec 21, 2016 19:46 UTC (Wed) by jkingweb (subscriber, #113039) [Link] (4 responses)

As the lowly administrator of a home server and a newbie to systemd, I was unaware of these features, but I can quickly see that, as a part of defence in depth, it makes nothing but sense to use them wherever practical. I'll definitely be doing some testing on those services where I've had to adapt my own unit files, and I'll be very curious to see any upstream takeup for those services where I use stock unit files.

Using systemd for more secure services in Fedora

Posted Dec 21, 2016 20:06 UTC (Wed) by drag (guest, #31333) [Link] (3 responses)

I definitely think that systemd unit files should be treated as source code in upstream.

One of the chief benefits of systemd is that provides significant unification between distributions. I should be able to write a unit file on Fedora for a normal everyday service and not have any hesitation that it'll work on any other Linux OS with similarly aged version of systemd.

If the Fedora devs can figure out more secure unit files then it seems like it would be a easy win for upstream to adopt them so that all Linux users benefit from them.

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 19:10 UTC (Thu) by johannbg (guest, #65743) [Link] (2 responses)

Well not all upstream wants to play the init system game on *nix and reject any such proposals be it systemd, sysv or something else, then there are upstreams that embrace systemd and upstreams that dont and the fact is that there is still a long way to go before type units can become universal on all linux distribution.

For that to happen alot of needless distribution deviation needs to vanish from the core/base OS level which corporates see as a threat or to disruptive to it's customer base and their administrators ( like the removal of /etc/sysconfig/foo environment files ) and community driven distributions see as a threat to their "individuality" since they no longer would be differentiating themselves from one another ( at least not on that level ).

Then some upstreams might be shipping 4 different parent types of type units to cater to it's target audience in which they need to be generic themselves for their application or application stack and rely on administrators to tweak the type units to fit their running environment.

Those upstream that ship type units are already way to slow to adapt and keep up with rate of change in systemd and the features ( security or otherwise ) it brings as is ( even if upstream is only carry a single type unit ) so an different approach to upstream carrying type unit is needed if the intent is for all to benefit all the features systemd has to offer.

You will most certainly not get that from Fedora or it's package maintainers ( which are often confused with developers or development of anykind ).

The distribution you will most likely see such effort come from ( if at all ) will either be Arch or some embedded/arm based one which are slowly overtaking the traditional pc distribution model and market.

Using systemd for more secure services in Fedora

Posted Dec 29, 2016 19:45 UTC (Thu) by drag (guest, #31333) [Link] (1 responses)

You are making a huge number of unfounded assumptions about the motivations of people involved in these sorts of things.

Sure there are issues, but generally speaking 'upstream first' is the way to go.

Using systemd for more secure services in Fedora

Posted Dec 29, 2016 23:25 UTC (Thu) by johannbg (guest, #65743) [Link]

Bare in mind I dealt with majority packagers who shipped legacy sysv init scripts in Fedora as well as wide varity of upstream at that time as well ( a period of what 4 to 5 years ) as well as did cross collaboration work between distributions working with systemd so by all means explain to me which huge number of unfounded assumptions am I making?

The upstream themselves in which their attitude to upstream involvement or exclusion of init script of anykind?
Their slow rate adopting type systemd units for their components or keeping up with the rate of change in systemd?
( An approach which they can never do, for systemd or for any other component for that matter. )

The fact packager claiming themselves somehow being developers in the distribution ( which probably has something to do with the fact at one point in history those that did downstream package maintenance where upstream developers and somehow managed to call themselves developers in downstream distributions ) when majority of them for those what 15k components in Fedora these days barely can debug their own component and full fill their role as distribution maintainers and acting as a liason between downstream and upstream?

Fedora being first in anything other than dressing itself in emperors clothes or having the PLL leading the community in circles. that's a good laugh well except for the Gnome half of the Red Hat Desktop team continuously breaking existing users setups but then again Gnome has been in continues beta state since I started using Gnome in Red Hat Linux 6.x somewhere around the year 2000 so technically is not a change or a feature, it's more like expected or tradition at this point in time.

"Generally speaking" concept like 'upstream first' cannot work with type systemd units until the fragmentation that exists in the downstream core/baseOS level seize to exist and I was personally saving one of such fight with the Red Hatters until I was done with the migration and would be doing the required clean up process to get the entire distribution on par with the current state of systemd at that time.

Seizing the fragmentation on the core/baseOS level is met with two opposition the corporate half ( Red Hat,Suse,Canonical ) who's having pissing contest on each other to gain market share thus "have" to deviate from each other in the process on that level and downstream distributions who think Linux is about choice and feel their freedom is threaten if someone change or do so differently from them.

The exact same mess and or quality depending how you view those scripts as was in legacy sysv init scripts has emerged upstream in systemd type units as in people that have no understanding of how type units work, do monkey sees, monkey copies and monkey pushes upstream type submitting, have pushed type units upstream, in which upstream which has no idea how systemd works thus cannot reliably review them, let alone fully understand what it's capable of and accept them relying on the "expertise" of the submitter.

Using systemd for more secure services in Fedora

Posted Dec 21, 2016 20:16 UTC (Wed) by roc (subscriber, #30627) [Link] (7 responses)

At least superficially it seems like it would be better to use a whitelist instead of a blacklist when defining a sandbox.

Using systemd for more secure services in Fedora

Posted Dec 21, 2016 20:26 UTC (Wed) by davidstrauss (guest, #85867) [Link] (6 responses)

> At least superficially it seems like it would be better to use a whitelist instead of a blacklist when defining a sandbox.

We (systemd) generally provide both options for each sandboxing directive, usually with an operator to invert black vs. whitelist.

Using systemd for more secure services in Fedora

Posted Dec 21, 2016 20:35 UTC (Wed) by roc (subscriber, #30627) [Link] (5 responses)

No, I mean instead of defining restrictions on one kernel feature at a time, with unmentioned features defaulting to "allow", deny access to *all* kernel features by default.

This is actually what the Chromium and Firefox sandboxes do; their seccomp filters whitelist syscalls, so any syscall that they don't know about is blocked.

Using systemd for more secure services in Fedora

Posted Dec 21, 2016 21:23 UTC (Wed) by davidstrauss (guest, #85867) [Link] (4 responses)

> No, I mean instead of defining restrictions on one kernel feature at a time, with unmentioned features defaulting to "allow", deny access to *all* kernel features by default.

The problem with that approach is that new classes of sandboxing cannot be assumed to have an empty whitelist for existing units; it would break all existing configurations any time a new directive gets introduced. You would need configuration versioning with strange effects like assuming an allow-all rule for any options introduced in later updates, which would partly defeat the "on by default" approach.

It also wouldn't work well with kernel disparities; systemd can only offer the sandboxing options the kernel can go on to enforce. What happens if someone tests on an older kernel (CentOS) with the latest systemd configuration format (i.e. using the versioned configuration approach from above) and then tries to deploy the configuration on something like Fedora? You're left with unfortunate options like: (1) force admins to configure explicit allow-all rules for configurations supported by systemd but not the current kernel, (2) silently ignore sandboxing rules not supported on the current kernel, (3) accept breakage between disparate kernels, or (4) offer extensive, conditional configuration to handle opportunistic sandboxing.

The closest current feature in systemd is setting NoNewPrivileges=true for a service (or making that the default for your services). This directive applies widespread sandboxing to high-privilege, often-unnecessary operations.

Using systemd for more secure services in Fedora

Posted Dec 21, 2016 22:08 UTC (Wed) by roc (subscriber, #30627) [Link]

> The problem with that approach is that new classes of sandboxing cannot be assumed to have an empty whitelist for existing units; it would break all existing configurations any time a new directive gets introduced.

New classes of sandboxing are tricky, yes. I think you'd need to carefully divide sandboxing features into "new classes" and "features within classes" where units opt into new classes of restrictions, but each class uses a whitelist.

For example, "syscalls" would be one class, and you'd start with a "reject all" seccomp filter and have directives to extend the filter to allow certain features. "Mount namespace" would be be another class, and you'd start with an empty namespace and have directives to mount various filesystems into it.

The problem with a blacklisting approach is that a sandbox which allows access to new kernel features whenever the kernel is upgraded is inherently weak.

I appreciate that various practical issues, including the design of the Linux kernel itself, make this difficult to do properly.

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 4:30 UTC (Thu) by rahvin (guest, #16953) [Link]

The higher the security of the system the less usability the system has. Obviously the most secure computer in the world would be one that's locked in a vault guarded 24/7 with no way to input, no output and no connection to a network. Though totally secure the system would be useless.

The balance is to find the highest level of security you can without compromising usability. I agree with the grandparent that whitelisting is far more secure and you don't seem to disagree as you point to usability concerns as a reason this is unreasonable security. Though there could be complications with whitelisted sandboxing the security benefits would IMO outweigh the hit to usability. Over the years Linux has tended to favor usability over default security. For years most distributions didn't even come with a simple default firewall let alone a generally secure firewall and disabled services that aren't needed. As a comparison you have OpenBSD that in a default install starts with a highly restricted firewall and a single running service (SSH).

Though I don't think our community should move to the default level of security as OpenBSD as it would severely damage usability and make the system virtually impossible for new users to start up with I do wish Linux Distributions would favor security over usability where cases are encountered like the one discussed here. Sandboxing is a security measure at it's core, adding a security hole where features are automatically enabled unless blacklisted creates insecurity by default. Whitelisting abilities on the otherhand would start with secure and allow the user to poke holes in that security for usability. In addition the advantage of whitelisting is that the distribution/package/container/vm can provide secure by default whitelisting that enables only the desired abilities.

One of the key selling points of systemd is the standardization that allows upstream and distributions to write the configuration in a simplified easy to read and modify configuration. Whitelisting sandboxes would seem to fall right into that same selling point IMO even with the complications it could cause because it lowers the bar to better security by default.

Security is hard and security by default helps everyone.

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 6:40 UTC (Thu) by josh (subscriber, #17465) [Link] (1 responses)

> The problem with that approach is that new classes of sandboxing cannot be assumed to have an empty whitelist for existing units; it would break all existing configurations any time a new directive gets introduced. You would need configuration versioning with strange effects like assuming an allow-all rule for any options introduced in later updates, which would partly defeat the "on by default" approach.

That still seems like a sensible approach. You could have a version number, which you increase every time you add a new sandboxing feature, and units could declare a version number they work with. The sandbox would then treat every feature up to that version as off-by-default, and every subsequent feature as on-by-default.

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 7:26 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

I work on some projects which have this kind of thing. It's a maintenance nightmare. Because sometimes you want the new stuff if available, but still have a low minimum version, so projects end up with asking for the new stuff if available, but with the bad defaults of the older version. This seems like a sensible maintenance trade-off to me, but I'm also not a systemd developer either; maybe it'd be easier there too.

FireJail

Posted Dec 21, 2016 23:07 UTC (Wed) by linuxrocks123 (subscriber, #34648) [Link] (1 responses)

This is a really cool project related to sandboxing: https://firejail.wordpress.com/

It's a mostly seccomp-bpf-based sandboxing mechanism that works with a wide variety of programs, both server and desktop. It ships with config files for most common Linux programs and attempts to automatically choose the right config file based on the name of the program being run.

Firejail includes its own config files for everything, so users of Firejail will not run into the blacklist-versus-whitelist problem with the SystemD approach: updating Firejail will update all its config files for all programs it supports, so the latest and greatest sandboxing mechanisms added to the kernel will automatically be used by the newest Firejail versions in a way that doesn't break the programs.

It's a great example of a program that does one thing and does it well.

FireJail

Posted Dec 22, 2016 1:39 UTC (Thu) by pabs (subscriber, #43278) [Link]

Here is another one that uses light-weight virtualisation:

https://cappsule.github.io/
https://github.com/cappsule/cappsule-doc/blob/master/inte...

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 0:57 UTC (Thu) by spender (guest, #23067) [Link] (5 responses)

I'm curious how the seccomp-based enforcement would actually prevent anything like this while still allowing users to use strace without a very large caveat about seccomp and ptrace except on very recent kernels. I don't see evidence of that caveat in this article, so I'm guessing this is a nice heap of false security for a majority of people relying on it.

https://gist.github.com/thejh/8346f47e359adecd1d53

-Brad

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 1:28 UTC (Thu) by mattdm (subscriber, #18) [Link] (3 responses)

Brad, what does "very recent kernels" mean in the above (and, what's the change?)

Thanks!

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 1:50 UTC (Thu) by spender (guest, #23067) [Link] (2 responses)

Since these series of commits from June of this year:
https://lkml.org/lkml/2016/6/9/627

They seem to only be in 4.8 (which was released in early October) and above, were never marked for backporting to earlier kernels (I just confirmed they were not backported to the upstream 4.4), and it would require reworking of the code given how much churn the x86 arch in particular has undergone in the past year or so.

-Brad

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 13:04 UTC (Thu) by mattdm (subscriber, #18) [Link]

Thank you. From a purely Fedora-centric point of view, this is fine, since we are currently on 4.8.15.

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 18:23 UTC (Thu) by tixxdz (subscriber, #60564) [Link]

Hi Brad,

Thanks for the link, doc fix here: https://github.com/systemd/systemd/pull/4960 , there is the @debug set that one can also use.

Using systemd for more secure services in Fedora

Posted Dec 22, 2016 2:31 UTC (Thu) by roc (subscriber, #30627) [Link]

systemd needs to disable ptrace within its sandboxes, definitely.

Users could still use "strace -p" to attach strace to a service from outside the sandbox, and I suspect that's the most likely way to apply strace to a service anyway.


Copyright © 2016, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds