LWN: Comments on "Using systemd for more secure services in Fedora"

Using systemd for more secure services in Fedora

johannbg — Thu, 29 Dec 2016 23:25:08 +0000

Bare in mind I dealt with majority packagers who shipped legacy sysv init scripts in Fedora as well as wide varity of upstream at that time as well ( a period of what 4 to 5 years ) as well as did cross collaboration work between distributions working with systemd so by all means explain to me which huge number of unfounded assumptions am I making?

The upstream themselves in which their attitude to upstream involvement or exclusion of init script of anykind?
Their slow rate adopting type systemd units for their components or keeping up with the rate of change in systemd?
( An approach which they can never do, for systemd or for any other component for that matter. )

The fact packager claiming themselves somehow being developers in the distribution ( which probably has something to do with the fact at one point in history those that did downstream package maintenance where upstream developers and somehow managed to call themselves developers in downstream distributions ) when majority of them for those what 15k components in Fedora these days barely can debug their own component and full fill their role as distribution maintainers and acting as a liason between downstream and upstream?

Fedora being first in anything other than dressing itself in emperors clothes or having the PLL leading the community in circles. that's a good laugh well except for the Gnome half of the Red Hat Desktop team continuously breaking existing users setups but then again Gnome has been in continues beta state since I started using Gnome in Red Hat Linux 6.x somewhere around the year 2000 so technically is not a change or a feature, it's more like expected or tradition at this point in time.

"Generally speaking" concept like 'upstream first' cannot work with type systemd units until the fragmentation that exists in the downstream core/baseOS level seize to exist and I was personally saving one of such fight with the Red Hatters until I was done with the migration and would be doing the required clean up process to get the entire distribution on par with the current state of systemd at that time.

Seizing the fragmentation on the core/baseOS level is met with two opposition the corporate half ( Red Hat,Suse,Canonical ) who's having pissing contest on each other to gain market share thus "have" to deviate from each other in the process on that level and downstream distributions who think Linux is about choice and feel their freedom is threaten if someone change or do so differently from them.

The exact same mess and or quality depending how you view those scripts as was in legacy sysv init scripts has emerged upstream in systemd type units as in people that have no understanding of how type units work, do monkey sees, monkey copies and monkey pushes upstream type submitting, have pushed type units upstream, in which upstream which has no idea how systemd works thus cannot reliably review them, let alone fully understand what it's capable of and accept them relying on the "expertise" of the submitter.

Using systemd for more secure services in Fedora

drag — Thu, 29 Dec 2016 19:45:37 +0000

You are making a huge number of unfounded assumptions about the motivations of people involved in these sorts of things.

Sure there are issues, but generally speaking 'upstream first' is the way to go.

Using systemd for more secure services in Fedora

johannbg — Thu, 22 Dec 2016 19:10:51 +0000

Well not all upstream wants to play the init system game on *nix and reject any such proposals be it systemd, sysv or something else, then there are upstreams that embrace systemd and upstreams that dont and the fact is that there is still a long way to go before type units can become universal on all linux distribution.

For that to happen alot of needless distribution deviation needs to vanish from the core/base OS level which corporates see as a threat or to disruptive to it's customer base and their administrators ( like the removal of /etc/sysconfig/foo environment files ) and community driven distributions see as a threat to their "individuality" since they no longer would be differentiating themselves from one another ( at least not on that level ).

Then some upstreams might be shipping 4 different parent types of type units to cater to it's target audience in which they need to be generic themselves for their application or application stack and rely on administrators to tweak the type units to fit their running environment.

Those upstream that ship type units are already way to slow to adapt and keep up with rate of change in systemd and the features ( security or otherwise ) it brings as is ( even if upstream is only carry a single type unit ) so an different approach to upstream carrying type unit is needed if the intent is for all to benefit all the features systemd has to offer.

You will most certainly not get that from Fedora or it's package maintainers ( which are often confused with developers or development of anykind ).

The distribution you will most likely see such effort come from ( if at all ) will either be Arch or some embedded/arm based one which are slowly overtaking the traditional pc distribution model and market.

Using systemd for more secure services in Fedora

tixxdz — Thu, 22 Dec 2016 18:23:02 +0000

Hi Brad,

Thanks for the link, doc fix here: https://github.com/systemd/systemd/pull/4960 , there is the @debug set that one can also use.

Using systemd for more secure services in Fedora

mattdm — Thu, 22 Dec 2016 13:04:55 +0000

Thank you. From a purely Fedora-centric point of view, this is fine, since we are currently on 4.8.15.

Using systemd for more secure services in Fedora

mathstuf — Thu, 22 Dec 2016 07:26:04 +0000

I work on some projects which have this kind of thing. It's a maintenance nightmare. Because sometimes you want the new stuff if available, but still have a low minimum version, so projects end up with asking for the new stuff if available, but with the bad defaults of the older version. This seems like a sensible maintenance trade-off to me, but I'm also not a systemd developer either; maybe it'd be easier there too.

Using systemd for more secure services in Fedora

josh — Thu, 22 Dec 2016 06:40:22 +0000

> The problem with that approach is that new classes of sandboxing cannot be assumed to have an empty whitelist for existing units; it would break all existing configurations any time a new directive gets introduced. You would need configuration versioning with strange effects like assuming an allow-all rule for any options introduced in later updates, which would partly defeat the "on by default" approach.

That still seems like a sensible approach. You could have a version number, which you increase every time you add a new sandboxing feature, and units could declare a version number they work with. The sandbox would then treat every feature up to that version as off-by-default, and every subsequent feature as on-by-default.

Using systemd for more secure services in Fedora

rahvin — Thu, 22 Dec 2016 04:30:55 +0000

The higher the security of the system the less usability the system has. Obviously the most secure computer in the world would be one that's locked in a vault guarded 24/7 with no way to input, no output and no connection to a network. Though totally secure the system would be useless.

The balance is to find the highest level of security you can without compromising usability. I agree with the grandparent that whitelisting is far more secure and you don't seem to disagree as you point to usability concerns as a reason this is unreasonable security. Though there could be complications with whitelisted sandboxing the security benefits would IMO outweigh the hit to usability. Over the years Linux has tended to favor usability over default security. For years most distributions didn't even come with a simple default firewall let alone a generally secure firewall and disabled services that aren't needed. As a comparison you have OpenBSD that in a default install starts with a highly restricted firewall and a single running service (SSH).

Though I don't think our community should move to the default level of security as OpenBSD as it would severely damage usability and make the system virtually impossible for new users to start up with I do wish Linux Distributions would favor security over usability where cases are encountered like the one discussed here. Sandboxing is a security measure at it's core, adding a security hole where features are automatically enabled unless blacklisted creates insecurity by default. Whitelisting abilities on the otherhand would start with secure and allow the user to poke holes in that security for usability. In addition the advantage of whitelisting is that the distribution/package/container/vm can provide secure by default whitelisting that enables only the desired abilities.

One of the key selling points of systemd is the standardization that allows upstream and distributions to write the configuration in a simplified easy to read and modify configuration. Whitelisting sandboxes would seem to fall right into that same selling point IMO even with the complications it could cause because it lowers the bar to better security by default.

Security is hard and security by default helps everyone.

Using systemd for more secure services in Fedora

roc — Thu, 22 Dec 2016 02:31:57 +0000

systemd needs to disable ptrace within its sandboxes, definitely.

Users could still use "strace -p" to attach strace to a service from outside the sandbox, and I suspect that's the most likely way to apply strace to a service anyway.

Using systemd for more secure services in Fedora

spender — Thu, 22 Dec 2016 01:50:50 +0000

Since these series of commits from June of this year:
https://lkml.org/lkml/2016/6/9/627

They seem to only be in 4.8 (which was released in early October) and above, were never marked for backporting to earlier kernels (I just confirmed they were not backported to the upstream 4.4), and it would require reworking of the code given how much churn the x86 arch in particular has undergone in the past year or so.

-Brad

FireJail

pabs — Thu, 22 Dec 2016 01:39:59 +0000

Here is another one that uses light-weight virtualisation:

https://cappsule.github.io/
https://github.com/cappsule/cappsule-doc/blob/master/inte...

Using systemd for more secure services in Fedora

mattdm — Thu, 22 Dec 2016 01:28:54 +0000

Brad, what does "very recent kernels" mean in the above (and, what's the change?)

Thanks!

Using systemd for more secure services in Fedora

spender — Thu, 22 Dec 2016 00:57:06 +0000

I'm curious how the seccomp-based enforcement would actually prevent anything like this while still allowing users to use strace without a very large caveat about seccomp and ptrace except on very recent kernels. I don't see evidence of that caveat in this article, so I'm guessing this is a nice heap of false security for a majority of people relying on it.

https://gist.github.com/thejh/8346f47e359adecd1d53

-Brad

FireJail

linuxrocks123 — Wed, 21 Dec 2016 23:07:54 +0000

This is a really cool project related to sandboxing: https://firejail.wordpress.com/

It's a mostly seccomp-bpf-based sandboxing mechanism that works with a wide variety of programs, both server and desktop. It ships with config files for most common Linux programs and attempts to automatically choose the right config file based on the name of the program being run.

Firejail includes its own config files for everything, so users of Firejail will not run into the blacklist-versus-whitelist problem with the SystemD approach: updating Firejail will update all its config files for all programs it supports, so the latest and greatest sandboxing mechanisms added to the kernel will automatically be used by the newest Firejail versions in a way that doesn't break the programs.

It's a great example of a program that does one thing and does it well.

Using systemd for more secure services in Fedora

roc — Wed, 21 Dec 2016 22:08:31 +0000

New classes of sandboxing are tricky, yes. I think you'd need to carefully divide sandboxing features into "new classes" and "features within classes" where units opt into new classes of restrictions, but each class uses a whitelist.

For example, "syscalls" would be one class, and you'd start with a "reject all" seccomp filter and have directives to extend the filter to allow certain features. "Mount namespace" would be be another class, and you'd start with an empty namespace and have directives to mount various filesystems into it.

The problem with a blacklisting approach is that a sandbox which allows access to new kernel features whenever the kernel is upgraded is inherently weak.

I appreciate that various practical issues, including the design of the Linux kernel itself, make this difficult to do properly.

Using systemd for more secure services in Fedora

davidstrauss — Wed, 21 Dec 2016 21:23:05 +0000

> No, I mean instead of defining restrictions on one kernel feature at a time, with unmentioned features defaulting to "allow", deny access to *all* kernel features by default.

The problem with that approach is that new classes of sandboxing cannot be assumed to have an empty whitelist for existing units; it would break all existing configurations any time a new directive gets introduced. You would need configuration versioning with strange effects like assuming an allow-all rule for any options introduced in later updates, which would partly defeat the "on by default" approach.

It also wouldn't work well with kernel disparities; systemd can only offer the sandboxing options the kernel can go on to enforce. What happens if someone tests on an older kernel (CentOS) with the latest systemd configuration format (i.e. using the versioned configuration approach from above) and then tries to deploy the configuration on something like Fedora? You're left with unfortunate options like: (1) force admins to configure explicit allow-all rules for configurations supported by systemd but not the current kernel, (2) silently ignore sandboxing rules not supported on the current kernel, (3) accept breakage between disparate kernels, or (4) offer extensive, conditional configuration to handle opportunistic sandboxing.

The closest current feature in systemd is setting NoNewPrivileges=true for a service (or making that the default for your services). This directive applies widespread sandboxing to high-privilege, often-unnecessary operations.

Using systemd for more secure services in Fedora

roc — Wed, 21 Dec 2016 20:35:03 +0000

No, I mean instead of defining restrictions on one kernel feature at a time, with unmentioned features defaulting to "allow", deny access to *all* kernel features by default.

This is actually what the Chromium and Firefox sandboxes do; their seccomp filters whitelist syscalls, so any syscall that they don't know about is blocked.

Using systemd for more secure services in Fedora

davidstrauss — Wed, 21 Dec 2016 20:26:14 +0000

> At least superficially it seems like it would be better to use a whitelist instead of a blacklist when defining a sandbox.

We (systemd) generally provide both options for each sandboxing directive, usually with an operator to invert black vs. whitelist.

Using systemd for more secure services in Fedora

roc — Wed, 21 Dec 2016 20:16:22 +0000

At least superficially it seems like it would be better to use a whitelist instead of a blacklist when defining a sandbox.

Using systemd for more secure services in Fedora

drag — Wed, 21 Dec 2016 20:06:58 +0000

I definitely think that systemd unit files should be treated as source code in upstream.

One of the chief benefits of systemd is that provides significant unification between distributions. I should be able to write a unit file on Fedora for a normal everyday service and not have any hesitation that it'll work on any other Linux OS with similarly aged version of systemd.

If the Fedora devs can figure out more secure unit files then it seems like it would be a easy win for upstream to adopt them so that all Linux users benefit from them.

Using systemd for more secure services in Fedora

jkingweb — Wed, 21 Dec 2016 19:46:53 +0000

As the lowly administrator of a home server and a newbie to systemd, I was unaware of these features, but I can quickly see that, as a part of defence in depth, it makes nothing but sense to use them wherever practical. I'll definitely be doing some testing on those services where I've had to adapt my own unit files, and I'll be very curious to see any upstream takeup for those services where I use stock unit files.