|
|
Subscribe / Log in / New account

The poisoned NUL byte, 2014 edition (Project Zero)

For those interested in the gory details of a complex exploit, Google's Project Zero page describes the process of getting arbitrary code execution from a single NUL byte written to the heap by glibc in an off-by-one error. "The main point of going to all this effort is to steer industry narrative away from quibbling about whether a given bug might be exploitable or not. In this specific instance, we took a very subtle memory corruption with poor levels of attacker control over the overflow, poor levels of attacker control over the heap state, poor levels of attacker control over important heap content and poor levels of attacker control over program flow. Yet still we were able to produce a decently reliable exploit! And there’s a long history of this over the evolution of exploitation: proclamations of non-exploitability that end up being neither advisable nor correct."

to post comments

Setuid inherits ulimit?

Posted Aug 26, 2014 13:50 UTC (Tue) by epa (subscriber, #39769) [Link] (10 responses)

From the article:
Well, what happens if we run pkexec again after running the shell commands ulimit -s unlimited and ulimit -d 1 ? These altered limits to stack and data sizes are inherited across processes, even setuid ones:
Ouch! And you can imagine setting the stack size limit to some tiny value might cause an exploitable crash in some setuid programs too. Is there any reason that suid binaries don't reset these limits to safe default values?

Setuid inherits ulimit?

Posted Aug 26, 2014 23:26 UTC (Tue) by zblaxell (subscriber, #26385) [Link] (9 responses)

It's not possible to know what a "safe" default value might be for a setuid program. Too small and attacks are not mitigated. Too large and new attacks (e.g. DoS) are possible. Too complicated and the mechanism for setting setuid process limits itself becomes exploitable.

Inadequately specified execution environment is one of the central reasons why setuid programs are such a bad idea.

Setuid inherits ulimit?

Posted Aug 27, 2014 11:08 UTC (Wed) by epa (subscriber, #39769) [Link] (8 responses)

Um, the safe value is any value as long as it's the same on all systems, including the systems used to develop and test the suid program. Just pick a set of numbers (such as the defaults used by your Linux distribution for ordinary userspace processes) and arrange for them to be set for all setuid binaries.
Inadequately specified execution environment is one of the central reasons why setuid programs are such a bad idea.
Exactly. The ulimit values are part of the execution environment, and need to be specified as part of what any setuid program can expect, rather than haphazardly inherited from the unprivileged parent process.

Setuid inherits ulimit?

Posted Aug 28, 2014 2:52 UTC (Thu) by zblaxell (subscriber, #26385) [Link] (7 responses)

There is no safe value because a value that is appropriate for one workload is inappropriate for others. Consider a busy web app hosting server running user code through cgi-bin. Normally CGI scripts would have tight resource restrictions, but they may also have access to a setuid sendmail binary. If this "default ulimit" scheme is implemented, it could result in a DoS opportunity instead of a resource starvation attack--doubly so if the default ulimit values come from the sendmail developer instead of the app hosting admin.

ulimit values form a relatively small part of the total attack surface that a child setuid process exposes to its unprivileged parent. If you specify ulimits for setuid program correctness then you have to deal with real resource exhaustion, prctl settings, and cgroup limits too. Failure to deliver precisely the resources specified by ulimit values under this scheme would result in a possible attack vector, so the semantics of ulimit must be significantly altered in a very expensive and inflexible direction under this scheme.

It is theoretically possible to specify the execution environment completely, but that specification is huge on a modern Unix, and implementing it correctly is invasive.

On Unixish OSes it's easier to just not bother with setuid in the first place. Linux has authenticated local IPC to allow unprivileged processes to collaborate with privileged peers when they need a temporary privilege boost. This capability alone makes most setuid use cases redundant.

Setuid inherits ulimit?

Posted Aug 28, 2014 11:02 UTC (Thu) by epa (subscriber, #39769) [Link] (6 responses)

I think you misunderstand my point. Perhaps I chose poor wording to call them 'safe' values for resource limits. I should have said 'standard' values. I quite agree that the appropriate limits vary by workload. Of course if the setuid program needs a higher limit then, being root, it can arrange to change the limit. The point is that the developer of the setuid program knows what to expect; you don't have the program developed and tested assuming some reasonable set of limits and then run with different limits. It always runs, by default, with the same standard limits the developer had in mind.

Would you ever want to run your setuid sendmail binary with lower resource limits? I see your point that resource restrictions are important, and you might want to apply them to child processes even if those children are setuid root. However, the premise of this discussion is that setuid programs can have bugs triggered by bumping against unexpected ulimit settings, so that running sendmail with lower limits might tickle a bug and end up causing an exploit. It's a matter of judgement in individual cases which is best.

I don't object to running setuid programs with funky ulimit settings, just as I don't object to running them with LD_PRELOAD plugins to muck around with the standard library. But I think that doing so should require an explicit and privileged step (so, for running sendmail, a suid root wrapper could set the lower ulimit values and then exec the sendmail binary). That is already the case for LD_PRELOAD, which is explicitly ignored for setuid binaries, and I think it should be the case for ulimit too.

You are right that there are many other bits of the execution environment to consider. That is not a reason to ignore this one particular bit. And you are also right that better alternatives exist for many cases where setuid binaries are used; but as long as setuid binaries continue to exist, they need to work in the safest and most predictable way possible.

Setuid inherits ulimit?

Posted Aug 28, 2014 12:18 UTC (Thu) by zblaxell (subscriber, #26385) [Link] (5 responses)

You still haven't provided a solution for the cgroups and real resource exhaustion cases, which have the same effect as the ulimit, but cannot be resolved by adding more privileged code to an already complex situation. In addition to all of the above, setuid programs have also been attacked through their inherited FDs (including working directory and TTY) and environment variables (including stuff you'd normally want to use from a privileged program, like TERM and TZ).

There are lots of surprising behavior differences that arise with setuid programs. This stuff has been analyzed to death for decades. Fast-forward to the end: you want the setuid program to behave exactly the same way as a program that was never setuid and never executed in the environment of an unprivileged parent in the first place, including--and especially--with respect to exploitable features of the execution environment that the privileged software developer might not be aware of. On Linux in particular, "features of the execution environment" is a rapidly changing and growing list.

It's much easier--and safer--to start a process from a safe, well-defined, and privileged environment than it is to get back there from an arbitrary starting point chosen with malicious intent.

My point is that we should put 'rootflags=nosuid' on the kernel command line, replace the handful of programs that break (e.g. sudo and mutt_dotlock) and never look back.

That setuid wrapper that sets up ulimits? Replace it with a server and a legacy-compatible client wrapper. setuid sendmail? Replace it with postfix. All those online tutorials that say 'sudo some-arbitrary-command'? Replace with 'ssh -XAt root@localhost some-arbitrary-command'. Need the child's working directory, stdin, or stdout, or just a safe way to open files as the unprivileged user? Pass file descriptors to the server process (we do have sendmsg(2) and fchdir(2) after all). And so on.

Setuid inherits ulimit?

Posted Aug 28, 2014 13:28 UTC (Thu) by epa (subscriber, #39769) [Link] (4 responses)

Here the program being attacked was pkexec which exists specifically to provide an authenticated way to execute a program as another user. (And that is a legitimate use case, as anyone who has worked on a support desk will attest.) While as you say, many examples of setuid programs can be replaced, not all can be.

Setuid inherits ulimit?

Posted Aug 28, 2014 13:59 UTC (Thu) by cortana (subscriber, #24596) [Link] (3 responses)

The replacement would be a service that runs programs, as requested via a client, as a particular user. A bit like ssh, but without the network/encryption overhead.

$ sudo find / -xdev  -perm /4000
/bin/fusermount
/bin/mount
/bin/ping6
/bin/ping
/bin/umount
/bin/su
/sbin/mount.nfs
/usr/bin/traceroute6.iputils
/usr/bin/chsh
/usr/bin/lppasswd
/usr/bin/sudoedit
/usr/bin/slock
/usr/bin/sudo
/usr/bin/gpasswd
/usr/bin/at
/usr/bin/X
/usr/bin/ksu
/usr/bin/nvidia-modprobe
/usr/bin/passwd
/usr/bin/staprun
/usr/bin/mtr
/usr/bin/chfn
/usr/bin/beep
/usr/bin/newgrp
/usr/bin/pkexec
/usr/lib/openssh/ssh-keysign
/usr/lib/pt_chown
/usr/lib/hadoop-yarn/bin/container-executor
/usr/lib/dbus-1.0/dbus-daemon-launch-helper
/usr/lib/chromium/chrome-sandbox
/usr/lib/hadoop-0.20-mapreduce/sbin/Linux-amd64-64/task-controller
/usr/lib/policykit-1/polkit-agent-helper-1
/usr/lib/virtualbox/VBoxHeadless
/usr/lib/virtualbox/VirtualBox
/usr/lib/virtualbox/VBoxNetDHCP
/usr/lib/virtualbox/VBoxSDL
/usr/lib/virtualbox/VBoxNetAdpCtl
/usr/lib/eject/dmcrypt-get-device
/usr/lib/x86_64-linux-gnu/spice-gtk/spice-client-glib-usb-acl-helper
/usr/sbin/uuidd
/usr/sbin/exim4
/usr/sbin/pppd
Some will be harder to replace with services than others.

Setuid inherits ulimit?

Posted Aug 28, 2014 14:01 UTC (Thu) by cortana (subscriber, #24596) [Link]

For the complete light show, try:

> find / -xdev -type f -perm /6000 -exec ls --color=auto -l {} +

Setuid inherits ulimit?

Posted Aug 28, 2014 18:31 UTC (Thu) by justincormack (subscriber, #70439) [Link]

Linux has had ping sockets for ages https://lwn.net/Articles/420799/ but distros still don't seem to be using them.

Setuid inherits ulimit?

Posted Aug 28, 2014 18:55 UTC (Thu) by zblaxell (subscriber, #26385) [Link]

Do any of those have to be setuid?

su, sudo, sudoedit, ksu, etc. can be replaced with a lightweight equivalent to 'ssh root@localhost' with various options. You can even reuse most of the exisitng sudo code unmodified (it just has to know that uid comes from SO_PEERCRED instead of geteuid()).

mtr, ping, traceroute6, etc. need a socket in raw mode. There are better ways that a process could get one, e.g. put permission bits on network interfaces in the kernel, so an admin can authorize any process to listen to the local WiFi interface without having completely unrelated root privileges. I gather OSX does something like this, so ping just works without setuid there.

Mail has long been available on Unix through a wide variety of servers that don't need setuid bits. Ditto user account management tools like chfn and passwd (NIS is well into decade #3, although a more lightweight replacement that isn't network-accessible and full of bugs might be a good idea).

User mounts and VM management tools are already available through dbus servers, so deprecate the legacy interfaces.

'at' and 'crontab' already communicate with a privileged server through an IPC (FIFO), so they're half done. Just implement the other half.

pkexec is...wrong. It already talks to a privileged server process for authentication, but it doesn't use that server process to provide the privileged process's execution environment. It's all right there, all it has to do is move some FD's around and fork()...*headdesk*

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 14:11 UTC (Tue) by imitev (guest, #60045) [Link] (76 responses)

Does someone know if grsecurity prevents that exploit ? BTW there's also no mention of selinux - disabled/permissive or in enforcing mode.

Off-topic: unrelated to that specific exploit, each time I'm reading such stories I think it would be a good thing to have a grsec kernel running on my internet facing centos (6x, 7x) servers.
It seems the only way to successfully apply the patches is on vanilla kernels, but some years ago I remember being bitten by subtle issues when running a vanilla kernel instead of the heavily patched RH one so I'm not too keen on trying again.
Q: Is there someone reading LWN running grsec in a RH environment willing to comment ? If yes, on vanilla or RH kernel ? If with vanilla, built from source (advised for randomization), or from a repo ?
Q': Is there anyone running vanilla kernels (without grsec) on RH 6x or 7x ?

thanks ! and sorry for the off-topic.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 15:38 UTC (Tue) by jamesmorris (subscriber, #82698) [Link]

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 15:52 UTC (Tue) by nnewton (subscriber, #40661) [Link] (1 responses)

I run grsec kernels on RHEL6 without much issue. Although the most recent kernel build has been bit by this: https://bugzilla.kernel.org/show_bug.cgi?id=60758

It definitely is a bit more work than running RH kernels, but when things like this come up it certainly seems worth the effort. We have specfiles here: https://github.com/orgs/el-grsecurity/dashboard, however, we are currently tracking down an issue with overflow detection.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 17:00 UTC (Tue) by imitev (guest, #60045) [Link]

Thanks !
Last time I've searched about centos/RH + grsec I didn't find that github page. I'll definitely try to build and test a kernel when time allows.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 16:23 UTC (Tue) by tialaramex (subscriber, #21167) [Link] (66 responses)

The default SELinux configuration has nothing to say about this bug. A setuid binary runs, the binary decides to start a chroot, and then it runs a root shell. Nothing prohibited by the out-of-box MAC there, SELinux isn't psychic and can't know that this was not intended.

I would argue that taking the lesson "I should use grsecurity" from this article is completely missing the point. The article's authors specifically set out to demonstrate that it's a waste of everybody's time to try to /guess/ whether things are exploitable and our best defence is to concentrate on _fixing_ and _preventing_ bugs instead. You'll notice that by the end they've relied on at least _two_ and arguably more distinct bugs. If any of those bugs were fixed the exploit fails.

Consider the Ubuntu "mitigation" mentioned at the start of the article. Should you switch OS to gain the protection of this "mitigation" ? It stops the exploit cold. But it's just a happy accident.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 17:26 UTC (Tue) by imitev (guest, #60045) [Link]

>> The default SELinux configuration has nothing to say about this bug.

One often reads exploit reports with "systems with X are not affected", with X often being grsecurity and to a lesser extent selinux. Hence my question about selinux, but you're probably right the default policy won't stop it (even if I wrote a few policies for some small daemons, I still have a hard time using selinux for everything but basic purposes).

>> our best defence is to concentrate on _fixing_ and _preventing_ bugs instead

While I wholeheartedly agree on "_fixing_" bugs, the sad fact is that not everybody does that (eg. lack of people reviewing kernel code). So security solutions have a real value at "_preventing_" bugs from being exploited - provided of course they don't add bugs on their own - which is another story.

And no, I won't switch to Ubuntu just because it stops a given exploit. You're focusing on a single example, while I have a feeling from the various exploits news I read that grsecurity *often* stop them. It's already a few times over the *years* that I've thought I should try it, but never really had the time to.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 17:36 UTC (Tue) by ewan (guest, #5533) [Link] (59 responses)

"taking the lesson "I should use grsecurity" from this article is completely missing the point. The article's authors specifically set out to demonstrate that it's a waste of everybody's time to try to /guess/ whether things are exploitable"

Neither grsecurity nor SELinux take the approach of trying to guess whether specific bugs are exploitable though; both attempt to block whole swathes of badness however it is provoked.

"and our best defence is to concentrate on _fixing_ and _preventing_ bugs instead"

An approach of "don't have bugs" is solid in theory, but I'm not sure it's terribly feasible in practice.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 18:59 UTC (Tue) by roskegg (subscriber, #105) [Link] (56 responses)

A great starting point: dump systemd. Fire Lennart and Kay. Don't let them work in Linux ever again. Those two guys have created more attack surface and chaos than anyone else in recent history.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 19:14 UTC (Tue) by mpr22 (subscriber, #60784) [Link]

Lennart lives in a country whose laws provide employees with certain rights regarding termination of employment. Given that there is, shall we say, not universal agreement that he is the dangerous incompetent you paint him as, dismissing him would require a substantive paper trail.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 19:50 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (11 responses)

What systemd has with any security problems?

You seem to be one of the old "we've always done it this way so that's why it's so" guard...

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 7:23 UTC (Wed) by roskegg (subscriber, #105) [Link] (10 responses)

I'm very open to new things. I'd like to see s6, the successor to daemontools, as the new init. It is in the spirit of the Unix Philosophy.

I am not opposed to new technology, new functionality. Extensions are fine. Breaking the fundamental social contract, that is not fine. Lennart and friends have been destroying the Unix social contract and pouring in all sorts of abominations from the Macintosh and VMS/Win32 environments. I specifically adopted Linux to get away from such.

All the GUI-clicky candy that MacOS has, can be easily implemented in the Unix spirit, as we saw with NeXT back in the day. We don't have to adopt their way of doing things, just to get a few of the User Interface features they have.

I've compared Linux to the Frankenstein monster in the past; it is becoming more and more apparent that my joke was too close to the truth. BSD is more like a well-knit Adonis.

Pulseaudio was a big botch; BSD provides proper audio drivers, so the need for Pulseaudio was never felt.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 7:38 UTC (Wed) by roskegg (subscriber, #105) [Link] (5 responses)

To clarify: fixing things that are broken, making things smaller and simpler, are all good. Such changes fit the Unix philosophy. Also the FORTH philosophy as outlined by Jeff Fox before he passed away (may he rest in peace)

http://www.ultratechnology.com/forth.htm
http://www.ultratechnology.com/forth0.htm

Changing the social contract is very different; the changes I accept are like the change in presidents, senators and congressmen every four years. The changes systemd and pulseaudio bring to the table are more like Taliban storming the White House, setting the US Constitution on fire, and telling people they can like it or get out of the country.

Lennart has a history of LYING. Example, 3 years ago he wrote this:

QUOTE
From all these talks I figured out that the Unix philosophy is:
1) Everything was already invented and perfect 40 years ago.
2) Everything else must be implemented using shell scripts and pipes.
QUOTE

That is nothing like the Unix philosophy, so ably outlined by Eric Raymond and Mike Goncarz. With this type of abusive language, why trust Lennart? He's a bag of wind. Worse, he is destroying Linux as a "better Unix", instead he is turning it into a "crappy MacOS/Win32".

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 13:37 UTC (Wed) by jwakely (subscriber, #60262) [Link] (4 responses)

How is your quote an example of LYING? And you think that's abusive? Have you read the bile you come out with? Do you need to have a lie down?

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 20:47 UTC (Wed) by roskegg (subscriber, #105) [Link] (3 responses)

Learn what the Unix Philosophy is, and you will know what type of lie it is. Once you know the Unix Philosophy, and see how Lennart and Kay are destroying it by shoving their software into all the Linux distributions, you will understand how extremely abusive their behavior is. Individual bits of software are replaceable. The living transmission of "how to go about doing things so we don't shoot our foot off" is not.

I already referenced Mike Goncarz book, "The Unix Philosophy", and Eric Raymond's book, "The Art of Unix Programming". Eric's book is even available for free online. I would also add to that Kernighan and Pike's book "The Practice of Programming".

http://www.catb.org/esr/writings/taoup/

This short article will give you some ideas:

https://pappp.net/?p=969

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 21:26 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Seriously, if so called 'Unix philosophy' leads to beautiful and concise code like this: http://anonscm.debian.org/cgit/users/lamont/bind9.git/tre... then it deserves to die.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 22:15 UTC (Wed) by intgr (subscriber, #39733) [Link]

> Learn what the Unix Philosophy is, and you will know what type of lie it is.

This is quite childish. There is a difference between a "lie" and an "opinion you disagree with".

And Lennart himself points out many things about systemd that are true to Unix tradition on his blog here: http://0pointer.de/blog/projects/the-biggest-myths.html (point #10)

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 11:31 UTC (Thu) by jwakely (subscriber, #60262) [Link]

I've read TAOUP and Kernighan & Pike, more than once.

> Learn what the Unix Philosophy is, and you will know what type of lie it is.

Ah, so it's the type of lie that is an opinion that you disagree with. Gotcha.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 9:01 UTC (Wed) by niner (subscriber, #26151) [Link] (3 responses)

What Unix social contract are you talking about? The only thing I find when searching for Unix social contract on Google are your postings on lwn.net mentioning it. So as it seems to be something you invented, would you care to explain what it says and who is supposed to sign it?

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 20:56 UTC (Wed) by roskegg (subscriber, #105) [Link] (2 responses)

Guess I made a mini-meme; the Unix Social Contract aliases directly to the Unix Philosophy, which is outlined in these three books:

* The Unix Philosophy, by Mike Goncarz
* The Art of Unix Programming, by Eric S. Raymond
* The Practice of Programming, by Kernighan and Pike (inventors of Unix)

The Linux social contract is even simpler; Linus Torvalds personally promised that Linux would be a gold standard POSIX implementation. That is why he went with X11 and NFS, even though those technologies are problematic. He made a promise that his goals were compatibility, and not breaking stuff that works.

You can innovate and move forward without breaking things that work. Linus did. Lennart isn't. You can fix things that are broken without replacing them. Think of babies and bathwater.

The purpose of the Unix Social Contract is to let programmers work together in the most effective way, without stepping on each others toes and surprisingly each other too much. Except for pleasant surprises.

Plan 9 innovated a lot, and even changed things, but it didn't violate the Unix Social Contract.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 23:35 UTC (Wed) by intgr (subscriber, #39733) [Link]

> Linus Torvalds personally promised that Linux would be a gold standard POSIX implementation.

Interesting, where did he state that promise? I believe he's been pragmatic about not supporting things that don't make sense in POSIX.

The oft-cited promise Linus made was about stability of Linux user space APIs from one version to the next, not compatibility with other systems.

> That is why he went with X11

Linus wrote the kernel, others worked on porting GUIs to Linux. XFree86 was ported because there wasn't much choice, non-X11 graphical systems were proprietary at the time. It had nothing to do with promises of compatibility.

> You can innovate and move forward without breaking things that work.

You mean just like PulseAudio adopted ALSA APIs unchanged, and how systemd provides transparent compatibility with sysvinit APIs (initctl, telinit etc) and initscripts including LSB headers; journal passthrough to syslog, and more?

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 31, 2014 19:05 UTC (Sun) by flussence (guest, #85566) [Link]

Please stop making grandiose proclamations, putting words in other people's mouths, and then gish-galloping away to some different comment thread when challenged to back your words up with independently verifiable facts.

If you respond to me with more of the same self-absorbed tripe you can expect a swift plonk.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 19:50 UTC (Tue) by Karellen (subscriber, #67644) [Link] (8 responses)

Yeah, because the sysv-init approach is *so* much neater and well-contained.

How does that go again? Oh, right, it consists of running mostly copy-and-pasted shell scripts, which import/includes the not-always well understood LSB init-functions, and depend on start-stop-daemon (or whatever similar helper your distro provides), and are possibly infected by other init script "helpers" from a nightmare beyond the abyss, and runs *all* of this as root. On top of that, sysv-init still can't keep proper track of what your damons are actually doing, and doesn't provide any simple ways of configuring things like ulimits or capabilities or private temp directories or read-only /usr or a whole bunch of other things.

I've heard many arguments for sysv-init over systemd, but I don't think "attack surface" ranks anywhere near the best of them.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 7:16 UTC (Wed) by roskegg (subscriber, #105) [Link] (7 responses)

Who said anything about sysvinit? If that is the only init you know, you don't know enough Unix to know how bad systemd is.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 18:29 UTC (Wed) by Karellen (subscriber, #67644) [Link] (6 responses)

> Who said anything about sysvinit?

99% of everyone else who's ever bad-mouthed systemd and the shadowy cabal Lennart has gathered together to subvert every single main distro in order to destroy their freedom.

> If that is the only init you know, you don't know enough Unix to know how bad systemd is.

Nope, I'm perfectly aware of upstart, launchd, rc, and runit. It's just that most systemd haters don't like them either, and are largely being mindlessly overprotective of their sysvinit knowledge and experience, and seem to mostly want to prevent that going to waste or becoming obsolete at any cost.

I was not aware of perp or s6 until just now though. And s6... actually does look interesting.

I think the main page needs a bit of an overview of how s6 fits together though. The "why" page spends a lot of time pointing out what all the other init systems get wrong, and the s6-svscan and s6-supervise are good in a man-page sort of way at describing tersely what those main parts of s6 do, but there doesn't seem to be a good introduction to the s6 system as a whole. (Unless I've missed it?)

For instance, I'm not sure how s6 handles dependencies between services. It seems like the most likely place for that to be addressed is the s6-supervise docs, but it's not. Nor in s6-svscan. s6-svwait seems to be a promising primitive for dependency resolution, and libftrigr seems to have something to do with it, but I can't quite figure out how these work together.

I do get the impression that s6 expects services to handle dependency resolution themselves, in their own processes (with the help of libftrigr?). While this might be a clean solution (or might not, I haven't considered it carefully enough yet) the reality of Unix programs today is that many do not handle dependencies natively, and rely on the init system to do this for them.

Therefore, if s6 does not handle inter-service dependencies (and if this is wrong, please ignore the rest of this paragraph, and forgive my ignorance), then either it needs to be patched to do so, or *all* the services one wishes to run need to be individually patched to manage their dependencies instead, or s6 will simply not be a viable replacement init on a system that needs to run existing Unix programs.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 15:02 UTC (Thu) by skarnet (guest, #98583) [Link] (5 responses)

I am the author of s6.

s6 is a process supervision system. Its goal is to launch and supervise daemons, and provide a framework to control them, as well as a framework for notification.

s6 is not an init system as most people envision it - not yet. It's definitely one of the main components for a full-featured init system, and I have used s6-svscan as my process 1 for a long time, but it's not a drop-in replacement for an init system as desktop users would define it.

It is an explicit non-goal of s6 to perform dependency management. Dependency management can be performed *on top* of a process supervision system, and I plan to add features to make it easier to use s6 as a backend for dependency management systems such as, for instance, OpenRC. I might design and implement my own dependency management system in the future, but since alternatives exist that people seem happy about, it is not a priority.

The "why" page of the s6 documentation is a bit dated. It was written when Upstart was the big thing, and systemd was added to it as an afterthought. I did not realize at the time how big systemd would become, and it is a critical failure on my part; I am going to address that failure. I fully acknowledge that the "why" page is inaccurate, and I should update it, but I have bigger plans.

In the near future (count, say, two or three months), I will be able to fully focus on developing a complete init system, based on s6, to provide an alternative to systemd - but I want to do it the *right* way, correctly engineered for reliability and smallness and following the Unix philosophy. The first step will be to reach out to users, and ask them what the killer features of systemd are, according to them - why they chose it over, for instance, sysvinit. I will then try to implement those features in a way that does not require you to hand over your machine, and your soul, to me.

There is still a long way to go. Attacking systemd advocates, and bickering in forums, accomplishes nothing: people use systemd because it scratches an itch for them, and that itch must be acknowledged, and addressed. It is definitely possible to do a better job than systemd; I believe s6 already does, on the parts that it's focused on, and I'm committed to keep working on it and eventually make it a serious competitor. But it will take time, dedication, and work. If you (as in everyone reading this) want to help, please stay tuned. I definitely encourage you to take a look at supervision systems in general and s6 in particular, as well as think about what exactly should go into an init system.

Thank you.

s6 response

Posted Aug 28, 2014 15:10 UTC (Thu) by karath (subscriber, #19025) [Link]

Nice update. All the best!

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 21:05 UTC (Thu) by anselm (subscriber, #2796) [Link] (3 responses)

Yours is a very laudable goal, and I wish you all the best in your endeavours.

Unfortunately though, systemd these days is much more than an init system. It offers standardised and generally reasonably well-designed replacements for huge swathes of base functionality which Linux distributions used to have to provide themselves, and which they generally provided in an insular and non-standardised fashion. As such, systemd has the potential to (a) save Linux distributors loads and loads of drudge work and let them concentrate on more interesting and important issues, and (b) make Linux that much easier to understand and deal with for administrators, instructors, and authors of books and training manuals, no matter what distribution they're using. Both of these are great and important benefits.

Anybody who thinks they will be able to supplant systemd by providing (only) a better »init system« will probably find that the major distributions are reluctant to buy into that, because the non-init features of systemd are too compelling to pass over, and are likely to become even more so in the future. This may or may not be an actual problem depending on who the target users of your replacement init are, but it is fairly safe to say that if you want to take over the Linux world it's not going to happen unless you get the mainstream distributions on board, and they won't take you seriously unless you offer something that improves on all of systemd.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 21:40 UTC (Thu) by skarnet (guest, #98583) [Link] (2 responses)

Yes, I realize that, and it will precisely be a major point of focus in my future efforts. What kind of "base functionality" does systemd provide that distributions rely on ? What are the killer features that were the tipping point for mainstream distributions and made them switch over to systemd ?

Of course the scope of the project goes well beyond an init system. That is exactly the problem with systemd - it aims to do everything, to be all-encompassing, and it is a political issue as well as a technical one. My goal is to provide a piece of software for every useful feature of systemd, every piece being as independent and replaceable as possible - one functionality, one tool. It will require a lot of study and design: I'm confident in my ability to do that.

I don't want to take over the world. I just feel that systemd is doing the right things the wrong way, and I want to provide an alternative. If the alternative is correctly designed, more and more people will choose it, and world domination will happen naturally. ;)

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 29, 2014 3:18 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

Have you seen the list of interfaces systemd uses, provides, and guarantees compatibility for? Could you please adhere to those at least (or work the get the interfaces improved if they are somehow insuffcient)? Even OpenBSD is implementing their own tools which adhere to them (at least timezoned and hostnamed, but there were at least two others in the list).

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 31, 2014 1:36 UTC (Sun) by johannbg (guest, #65743) [Link]

Writing a new init system is one thing, convincing wide variety of upstream maintainers and provide patches to their components is another and getting distribution to default, integrate and use it is the third.

Canonical failed in that aspect in every way with upstart so I somehow doubt that you will succeed in that regard in fact the only init system beside system V to achieve that, out of all the init system in existence out there past and present is systemd.

And despite what you might think that did not happen because of one man. An man that people seem to be blaming and basically hunting like captain Ahab hunts his white whale in Herman Melville's Moby-Dick based on some religious notion and misinterpretation of guidelines Doug McIlroy,Rob Pike,Ken Thompson came up with sometime in the 70's when times and technology where different.

The fact is today there are close to 200 people actively contributing to systemd and of those ca 200 people, 10% - 20% have direct commit access to it's source repository and that's just the people in the systemd's community that commit code, that's not counting the people that reside there and are actually doing all the heavy lifting and integration work outside and within the distribution themselves. The migration,documentation,teaching etc.

Bottom line you cannot achieve this alone and expect about 5+ years of full dedication to your project before it starts taking of and another 5 - 10 years until distribution and upstream will even consider going through the pain point of replacing init system again and upstream to rewrite their code ( under the assumption you manage to build a community surrounding your project ) but good luck we welcome and need the competition to prevent ourselves from stagnating as time change and technology evolves.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 20:44 UTC (Tue) by anselm (subscriber, #2796) [Link] (14 responses)

Specific examples, please.

(It's not as if the loads of mostly unstructured and inconsistent stuff that systemd replaces are entirely without their own problems. And moving much of the starting-a-service-in-the-background, socket-activation, resource-control, … machinery into systemd – where it can be implemented, debugged, and documented once and its configuration can be consistent across different services –, instead of forcing every single service to implement and debug it over and over again, presumably with old and new bugs, can also only be a good thing in the long run. In effect this actually reduces the attack surface.)

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 7:28 UTC (Wed) by roskegg (subscriber, #105) [Link] (13 responses)

The mind boggles. I don't think you understand how init systems work, or what they are for. Or even the Unix philosophy, or concepts like layering, coupling and cohesion. Have you read this?

http://skarnet.org/software/s6/why.html

Also please read the Unix social contract:

http://www.catb.org/esr/writings/taoup/

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 8:38 UTC (Wed) by anselm (subscriber, #2796) [Link] (12 responses)

While we're doing ad-hominem attacks, I don't think you know a lot about systemd. (The first thing you don't seem to know about systemd is that it is not in fact »just« an init system. The other thing you don't seem to know is that it is not in fact one huge monolithic blob of code. There are probably more.)

I'm actually just now teaching a class on systemd (among other things) and the great thing is that it really works. The participants like it. And I'm still waiting for your specific factual examples that explain how systemd increases the »attack surface« of a Linux system.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 21:28 UTC (Wed) by roskegg (subscriber, #105) [Link] (11 responses)

>While we're doing ad-hominem attacks, I don't think you know a lot about >systemd.
>The first thing you don't seem to know about systemd is that it is not in >fact »just« an init system. The other thing you don't seem to know is that >it is not in fact one huge monolithic blob of code.

That isn't an ad hominem, that is an outright straw man. Are you trying to have an honest discussion? I'm very aware of both those things you mention.

I notice systemd supporters use a lot of straw men, ad homimens, and outright misrepresentations. And then project these qualities onto the voices of sanity.

If systemd was so good, you wouldn't have to use these dishonest and coercive tactics.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 8:24 UTC (Thu) by anselm (subscriber, #2796) [Link] (10 responses)

If systemd was so good, you wouldn't have to use these dishonest and coercive tactics.

Actually, systemd is pretty good. At least it's good enough that all mainstream Linux distributions have either made it the default already or have declared that they will. This includes communities like Debian that don't generally let themselves be fazed by underhanded tactics such as the ones you're ascribing to the »systemd supporters«, and communities that used to have their own horse in the race, like Canonical/Ubuntu.

There is no way systemd could have ended up where it is now if it didn't present compelling technical advantages vis-à-vis the competition. Nobody claims that systemd is perfect, but on the other hand nobody has been able to point out specific architectural problems other than the vague waffling along the lines of »It's not Unix!« that one tends to get from the likes of you. (Note that you still need to provide a few specific examples in support of your claim that systemd increases the »attack surface« of a system.) In fact, there are various testimonials from people who used to argue vehemently against systemd but then after actually having had an in-depth look at it decided that it was in fact quite good. It is also very instructive to study the deliberations of the Debian technical committee on the subject.

Systemd detractors will have to learn to live with the fact that the caravan has moved on. Three years from now we will look back and wonder what all the flap was about, and how something like the earlier SysV init/inetd/cron/… kaboodle could ever have lasted as long as it did. This is not to say that systemd is the last word in its area – as the Upstart camp has had to find out to their disappointment, there is always room for something noticeably better. Arguably with systemd, the bar for »something better« has been raised considerably but if systemd is really so bad (and nobody noticed) and other solutions are so good, it shouldn't be a problem for you – or like-minded folks – to present an alternative that is technically so vastly superior that people will move over in droves.

systemd has some *serious* problems

Posted Aug 28, 2014 11:12 UTC (Thu) by dps (guest, #5725) [Link] (5 responses)

I have some very negative experience of systemd, like it failing badly by attempting to mount a NFS file system before starting networking. The only way I know of fixing that is to use "init=/bin/bash" and have a shell on another VT to poke systemd with so it actually works, and that only gets me a small value of works.

If I had system V init or even old style *BSD init scripts solving this problem, without use of init=/bin/bash, would be trivial. As it is I have not managed to solve it in a reasonable way.

My stripped down firewall machine neither needs nor wants anything within a mile radius of dbus and that includes systemd. Quite a few heavy servers also actively don't want dbus. While systemd may be great if it works and the box is a full feature desktop it fails the "can I fix it?" and "can I use on a stripped down system?" tests.

systemd has some *serious* problems

Posted Aug 28, 2014 11:43 UTC (Thu) by imitev (guest, #60045) [Link]

re - nfs problem: did you file a bug or asked for help ?

re - stripped down machine: there's a bunch of people who seem very happy with running systemd on embedded system. What doesn't fit with your stripped down setup (I'm truly asking, I don't doubt you have issues) ?

systemd has some *serious* problems

Posted Aug 29, 2014 19:11 UTC (Fri) by jspaleta (subscriber, #50639) [Link] (3 responses)

hmmm nfs mount attempts before starting networking...

Do you mean before network is actually fully up but after the network service has run? That's probably want you mean. As network service starting is not the same as networking being fully up and operational in a highly paralleled init.

If you need to run network services that can't cope with the network not being online then you should look at enabling the
NetworkManager-wait-online.service which will attempt to wait for the network to be online.

If you aren't running NetworkManager, then you can create a systemd service definition which mimics the NetworkManager-wait-online.service structure and replace the ExecStart line with a call to whatever executable you want to use to replace the nm-online call that blocks until the network is fully up, however you want to test for that. If you just want to block for 1 minute... you could do that.

Many network daemons actually init perfectly fine without the network being fully up, and waiting for the network to come up drastically slows down the boot time for many usage cases like say wireless laptops. The NetworkManager-wait-online service is provided, but turned off by default (by probably all distributors, but I haven't tested all distributions..)
as a compromise. Server admins should can self-assess whether they need to enable the wait service based on the deployed workload.

All of this was sorted out in the Fedora 14 and Fedora 15 timeframe. Arch even has this documented in their wiki as well for NetworkManager. But that's the easy stuff.

For my laptop that need to interact with different nfs mounts only available on different networks. I use NM dispatcher scripts so that my nfs mounts only get attempted as part of NM managed network bring up/bring down when I connect/disconnect to specific networks. Works a treat. I dock the laptop at work, get on the wired network, nfs mounts come up automagically without me having to do anything extra. And the boot time isn't delayed unncessarily for network bring up. Obviously this won't work for a server workload that uses nfs mounts for critical filesystems, but for that workload the wait online service is the obvious solution.


systemd has some *serious* problems

Posted Aug 31, 2014 0:24 UTC (Sun) by johannbg (guest, #65743) [Link] (2 responses)

"All of this was sorted out in the Fedora 14 and Fedora 15 timeframe."

This is not correct neither NM nor NFS and it's units where sorted out in that timeframe.

NetworkManager-wait-online since it's appearance have either been turned on or off ( directly or indirectly ) depending on which Fedora release it and which target it got tied to. Currently ( F20 ) it's "enabled" which has it's downsides and due to those downside it was disabled in the firstplace.

NFS has been utterly broken all the way up to F19 which was the last time I checked and tested as well it was the time I gave up running after Steve's whims maintaining it in Fedora. Based on his actions he seemed to be determine keeping it broken in Fedora.

systemd has some *serious* problems

Posted Sep 2, 2014 21:41 UTC (Tue) by bfields (subscriber, #19510) [Link] (1 responses)

Note upstream nfs-utils has its own unit files (see nfs-utils/systemd/) now, written mostly by Neil Brown. I assume that's what recent F20 or rawhide is using by now, but haven't checked. I don't know what you considered broken before, but worth checking to see if it's fixed now.

systemd has some *serious* problems

Posted Sep 2, 2014 22:19 UTC (Tue) by johannbg (guest, #65743) [Link]

I'm pretty sure Neil Brown borrowed few bits from me and what kept and I consider broken in NFS in Fedora is not NFS itself it is it's Fedora maintainer. Heck I even spent 6 hours cleaning up the spec file for him.

No we could have ironed out any integration/migration issues between systemd and nfs in F16 but here we are and I guess Red Hat finally had to get it's act together when RHEL 7 got released and properly fix this.

The systemd migration/integration would be completed two or four releases ago in Fedora if it weren't for package maintainership ( or rather lack there of ) still only 80% done in sysv migration and at least 1000 man hours yet left to properly cleanup and integrate other aspect of systemd into the distribution.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 12:37 UTC (Thu) by jb.1234abcd (guest, #95827) [Link] (3 responses)

I have already posted here on systemd's attack on UNIX/Linux software
development model and its shared ecosystem, and systemd's faulty design and
implementation, after Debian's decision to accept it as their default init
system.

May I remind all apologists of systemd that it was supposed to be a replacement for previous (but still alive) init systems.

Here is an update on systemd for you.

Jul 2014
Lennart Poettering gave a talk recently in Beijing about the state of systemd and its future ahead.

Lennart keynoted at the joint FUDCon Beijing 2014 with GNOME.Asia 2014 event and he talked about the current position of systemd and its future going forward, while acknowledging it's evolved more than just being a basic init system to being "a set of basic building blocks to build an OS from."

The tasks mentioned that systemd already covers include, "init system, journal logging, login management, device management, temporary and volatile file management, binary format registration, backlight save/restore, rfkill save/restore, bootchart, readahead, encrypted storage setup, EFI/GPT partition discovery, virtual machine/container registration, minimal container management, hostname management, locale management, time management, random seed management, sysctl variable management, and console managment."

Tasks being worked on are support for a local DNS cache, mDNS responder, LLMNR responder, DNSSEC verification, IPC support in the kernel (KDBUS), time synchronization with NTP, better integration with containers, and many other services.

Well, I remember also that he assured everybody that the UNIX/Linux bazaar
development model will be respected in the future.

jb

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 12:47 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link]

Where did he assure that?

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 12:54 UTC (Thu) by anselm (subscriber, #2796) [Link]

Tasks being worked on are support for a local DNS cache, mDNS responder, LLMNR responder, DNSSEC verification, IPC support in the kernel (KDBUS), time synchronization with NTP, better integration with containers, and many other services.

Sounds great to me.

Well, I remember also that he assured everybody that the UNIX/Linux bazaar development model will be respected in the future.

At this point two things are probably worth mentioning:

  • Contrary to popular belief, Lennart Poettering is not the sole developer of systemd. In fact there is now a fairly large and active development community around it, including people affiliated with many different Linux distributions. This helps ensure, at least, that systemd represents a consensus that most Linux distributors can live with, rather than the foibles of one single person. (Incidentally, nobody seems to mind that Linus Torvalds is still in charge of Linux.)
  • Also contrary to popular belief, systemd is not a single huge monolithic take-it-or-leave-it thing. It is good to see people working on standardised solutions for various problems within the Linux sphere under the systemd umbrella, because such solutions have a very good chance of actually becoming part of most Linux systems by way of the mainstream distributions incorporating systemd. For example, it would be very useful indeed to have a widely deployed DNS resolver that can verify DNSSEC, which right now distributions do not tend to offer by default. If in the future the verifying DNS resolver within systemd is not to people's liking, they will be able to either improve it or else replace it with another resolver that they like better, but whatever systemd comes with is likely to be at least a baseline implementation that will get the functionality into the hands of most Linux users without the need for them to sort this out themselves.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 13:26 UTC (Thu) by raven667 (subscriber, #5198) [Link]

Most of those examples are separate utilities that ship under the systemd umbrella which you can use or not use at your choice and most of those functions were handled by the initscripts in the previous SysV based system so there really hasn't been a change of where the functionality lives, even if you are using shell scripts you still need to be able to cover all those use cases.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 22:21 UTC (Tue) by drag (guest, #31333) [Link] (11 responses)

> A great starting point: dump systemd.

Yes, because the take home message we should get from this article is that depending on executing setuid binaries and ad hoc sudo-evoking scripts in a user's account to deal with routine/daily administrative tasks is a big win for security over automating things with specific purpose daemons over a well documented and standardized messaging protocol.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 7:45 UTC (Wed) by roskegg (subscriber, #105) [Link] (10 responses)

Solving those problems doesn't require systemd, with its bloat and violation of the social contract.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 7:49 UTC (Wed) by roskegg (subscriber, #105) [Link] (5 responses)

More to the point, the social contract reduces the attack surface at a fundamental level, by making it easy to trace out how the system works, debug it, and so on.

For an example of things done right, look at the Plan 9 operating system, and the 9P protocol. Perhaps 9P was the inspiration for D-Bus, but hot DAMN, how could Lennart manage to take something so nice and make it so ugly? Plan 9 is the successor to Unix, in the Unix philosophy, by the creators of Unix. Sweet operating system.

Composability is a fundamental property of Unix; Lennart doesn't respect composability of software units.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 10:25 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

I think if all those people whining about plan9 and the art of Unix programming at least made an attempt to write a reliable and easy to use system, then they would have had a point. As it is, by now cgroups and systemd are the only reliable pieces of infrastructure that is used to run tasks.

So basically, either own up or shut up.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 21:10 UTC (Wed) by flussence (guest, #85566) [Link] (3 responses)

> As it is, by now cgroups and systemd are the only reliable pieces of infrastructure that is used to run tasks.

Which version of cgroups is considered the reliable one? The one that's been available in a released kernel for approximately 3 weeks and requires a special developer mount option with lots of underscores and uppercase, or the one that's being phased out because it's considered an unmaintainable ball of mud by the very people who designed it?

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 21:28 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

Cgroups have always been reliable for simple process tracking. Only resource controllers have been problematic.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 23:20 UTC (Wed) by deepfire (guest, #26138) [Link] (1 responses)

So, the main functional extension cgroups provide is causing problems.. Hmm.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 23:29 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

We've been using cgroup controllers for several years. They do have some problems, but they are also incredibly powerful.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 10:23 UTC (Wed) by drag (guest, #31333) [Link] (3 responses)

> Solving those problems doesn't require systemd, with its bloat and violation of the social contract.

When the choice is using systemd versus day dreaming about a more perfect system while using the old way of doing things... I'll pick systemd, thank you very much.

Also violating a imaginary 'social contract', which appears to have been invented by you for commenting on this article, doesn't really seem much like a problem to me.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 21:18 UTC (Wed) by roskegg (subscriber, #105) [Link] (2 responses)

I feel a bit sad that you don't know about the social contract. It is valuable to both programmers and users. Never fear! Help is at hand. Read these three books: "The Art of Unix Programming", by Eric S. Raymond. "The Unix Philosophy", by Mike Goncarz. And "The Practice of Programming", by Kernighan and Pike.

There are many alternatives to systemd. openrc added cgroup support, for instance, and found that it wasn't hard or complex. Although not my first choice, openrc reflects the proper Unix Way. But I have high hopes for s6. s6 is a good evolution of daemontools. The BSD and Plan 9 init systems could always be ported over. The systemd alternatives are not daydreams; they are here now.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 7:57 UTC (Thu) by tao (subscriber, #17563) [Link]

Please enlighten me: if you're so upset about systemd, pulseaudio, bla bla bla <Lennart conspiracies bla bla bla>, then why aren't you using one of the BSD:s or Plan 9 instead? They're freely available, open source and not using any of the things that you so revile.

You could even switch to Solaris, though considering that SMF is one of the inspirations for systemd and that the ever so lauded ZFS is probably a blatant violation of your "UNIX social contract" what with its integration of multiple different layers that might not be what you'd like.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 8:56 UTC (Thu) by anselm (subscriber, #2796) [Link]

s6 may be great for the supervision of individual services but reading through its web pages it seems that s6 does not handle inter-service dependencies at all. That disqualifies it right out of the gate as a viable alternative to even the process-supervision part of systemd. And that is before considering features like resource control or child process tracking that systemd does out of the box, in a unified fashion across services, based on simple declarative configuration settings, but which users of s6 must provide piecemeal in their own script code as part of individual services' »run scripts«. (And s6 even comes with its own shell that people will have to learn to use.)

It is probably possible to come up with something that will shoehorn dependency management into s6 the way insserv has shoehorned it into SysV init. This will undoubtedly do wonders for keeping s6 simple and straightforward to understand. As it is, it looks as if there would have to be considerable work on s6 for it to be as good as systemd even on s6's own turf. It is not what I would view as a reasonable contender.

The poisoned discussion of some new projects

Posted Aug 27, 2014 14:47 UTC (Wed) by karath (subscriber, #19025) [Link] (6 responses)

I read this site for it's very high signal to noise ratio. This comment is both a non-sequitur to the original article and a hyperbolic and abusive attack.

This form of behaviour decreases the utility of this wonderful news site to me. Unfortunately, in the past few years, I've seen similar attacks on people who are trying to solve big problems in new projects, particularly systemd and Wayland.

I'd love to suggest that the answer is simple: either ignore the hated projects, hoping that someone else will come up with something better, OR join or create a project to solve the problem in an acceptable way and build a like-minded community around the project. On the other hand, I know that my hope, while not unreasonable, is deluded.

However, foisting one's irrational response to a changing world on a community filled with skeptical and questioning minds is doing little to support your cause and may well be acting as a recruitment banner to the opposition.

I've read through the later comments in this thread and read the referenced articles. I read http://skarnet.org/software/s6/why.html and, as far as I can tell, it has a serious error. The author assumes, because (s)he is familiar with upstart, that systemd is built in the same way. In particular the article assumes that systemd is a monolithic _process_ in pid 1. systemd may well be a monolithic _project_ but is made up of many executables running under a relatively minimal pid 1 executable.

Finally, having read and enjoyed reading many of the referenced articles, I'll ask the author of this comment: Have you read sufficient of the systemd documentation or even code so that you can make an informed attack? An informed attack is much more likely to convert others to your side than an uninformed attack.

The poisoned discussion of some new projects

Posted Aug 27, 2014 21:46 UTC (Wed) by roskegg (subscriber, #105) [Link] (5 responses)

>I'd love to suggest that the answer is simple: either ignore the hated >.projects, hoping that someone else will come up with something better, OR >join or create a project to solve the problem in an acceptable way and >build a like-minded community around the project. On the other hand, I >know that my hope, while not unreasonable, is deluded.

Think of network effects. Lennart not only looks like the young Bill Gates, but he is doing "embrace and extend" exactly the way Bill Gates did. Many of us programmers came to Linux to get away from that. I use systemd, and it feels like wearing handcuffs.

If I go my own way, I lose the network effects of sharing in a larger group. I could conceivably join the s6 project and help, but if no major distro adopts it, I don't have time and energy to maintain my own Linux distro.

Hard feelings? Hell yeah. Lennart is virally taking over the Linux ecosystem and turning it into a horrid mishmash of VMS/Win32/MacOS cruft.

In the early days, Gnome was very much designed in the Unix Philosophy. But over time it become more "Windowsy". *shudder* Originally, Gnome gave me a nice slick GUI as good as Windows and it was fast closing in on MacOS. And I could find the dot-files in my home directory and configure every app and tweak it to my hearts content. As much as I loathe C++, I have to say, KDE is far more in the Unix Philosophy now than Gnome. And C++ is renowned for opaque binary blobs! Something happened around 2007/2008, and Gnome has been downhill from there. The freedesktop guys have been trying to "fix" X11 for 10 years now. But they were the ones that designed it in the first place. I see them as the initial vector for this brain damage.

The Windows Registry must never be repeated.

The poisoned discussion of some new projects

Posted Aug 27, 2014 22:06 UTC (Wed) by rodgerd (guest, #58896) [Link] (3 responses)

> Lennart not only looks like the young Bill Gates, but he is doing "embrace and extend" exactly the way Bill Gates did

If you aren't going to seek the professional help you appear to so desperately need, please at least stop posting unless you can confine yourself to actual technially interesting material based in reality.

The poisoned discussion of some new projects

Posted Aug 27, 2014 22:19 UTC (Wed) by roskegg (subscriber, #105) [Link] (2 responses)

Come on, look at Lennart. You don't see the resemblance to the young Bill Gates?

The poisoned discussion of some new projects

Posted Aug 27, 2014 22:21 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Come to think about it, RMS looks a bit like Jesus.

That seals it. I'm switching to Emacs and joining the Church of Emacs.

The poisoned discussion of some new projects

Posted Sep 3, 2014 17:01 UTC (Wed) by jzbiciak (guest, #5246) [Link]

And it has exactly zero relevance. Depending on your opinion of Bill Gates, it could be considered an ad hominem argument as well.

The poisoned discussion of some new projects

Posted Aug 31, 2014 9:44 UTC (Sun) by nix (subscriber, #2304) [Link]

I clearly shouldn't feed the probably mentally disturbed individual, but...
And C++ is renowned for opaque binary blobs!
Is it? How? When did this happen?

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 21:52 UTC (Tue) by rgmoore (✭ supporter ✭, #75) [Link] (1 responses)

An approach of "don't have bugs" is solid in theory, but I'm not sure it's terribly feasible in practice.

I would say that decades of security experience show it isn't feasible. At the very least, it has never been done for anything remotely close to the size of a modern general-purpose operating system. Some kind of defect mitigation is the only reasonable response.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 11:13 UTC (Wed) by epa (subscriber, #39769) [Link]

"Don't have bugs" is impractical if it relies on a human programmer not to make mistakes. It can work if the language, compiler, and runtime are designed in such a way as to make a particular class of bug impossible (or at least impossible to introduce accidentally). For example, array accesses can be checked by default so that buffer overrun bugs no longer exist (unless the programmer has explicitly chosen to make a section of code unsafe).

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 23:07 UTC (Tue) by spender (guest, #23067) [Link] (4 responses)

*Shouldn't* the default SELinux configuration have something to say about the bug? I don't see anything in the description of pkexec that suggests it needs to be able to chroot, so why would the policy shipped with the distro (which we're lead to believe have been meticulously crafted by experts) unnecessarily grant the use of unnecessary capabilities and system calls?

SELinux doesn't need to be psychic to be able to do that -- it's a pretty basic function of an access control system, even more so for one that's supposed to have high granularity. A system with a policy automatically generated by grsecurity's full-system learning certainly wouldn't have permitted that chroot.

While your interpretation is close to what Google wants you to think from their showboating post, the fact that grsecurity prevented numerous techniques in the exploit in various ways can't be any more clear of a vindication of our security strategy. As the PaX Team says, fixing bugs reduces insecurity, but defending against exploit vectors and closing down entire bugclasses provides security.

-Brad

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 5:27 UTC (Wed) by wahern (subscriber, #37304) [Link] (2 responses)

This is why Capsicum is a much better long-term model for application security.

The knowledge about what privileges are needed and which can be dropped is _already_ known to the developer of the application.

Systems like SELinux appeal to sysadmins and database administrators, whose hammer is tweaking configuration files. And while that's often useful and necessary, it's no substitute for giving the developer the ability to manipulate privileges at the very places were privileges are used and applied.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 12:41 UTC (Wed) by raven667 (subscriber, #5198) [Link] (1 responses)

It's not an either/or situation, Capsicum is not the be-all, end-all solution to application security, it only helps with a subset of use cases, and is orthogonal to a permission checker like SELinux. Manipulating privileges from inside your program is great for the code you write yourself, but doesn't help for the vast majority of the code on your system which you didn't write. Specifying your own seccomp syscall filters and SELinux permissions are part of the whole solution as well.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 17:00 UTC (Wed) by ThinkRob (guest, #64513) [Link]

There are two types of access control questions to answer though:

1) What does this application need to do in order to perform all permutations of <insert functionality here>?

2) What does this application need to do in order to perform its role on <insert system here>?

#1 is what the developers can and should answer. Something like Capsicum is a good tool for them to provide that answer. The author of an image viewing application knows it's never going to need to list on a TCP port for anything, so he can make that declaration via some AC system and close off an entire class of attacks.

#2 is what system administrators answer. Something like SELinux or grsecurity's RBAC is a good tool for them to provide that answer. Some image viewer might allow for you to save a file to disk, but it's up to your system administrator to determine whether that's a "safe" action (and thus whether to categorically prevent it.)

You can generally express the answers to #1 using a tool suited for #2, but that's error prone, tedious, and generally not a good solution.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 8:00 UTC (Wed) by smcv (subscriber, #53363) [Link]

pkexec is a controlled-privilege-escalation tool, like su or sudo. It needs to be able to do whatever privileged things its user wants it to do.

If the SELinux approach to that is "su/sudo/pkexec still shouldn't be able to chroot(2), but non-setuid tools that actually need to chroot like chroot(8) or schroot(8) should be able to, and su/sudo/pkexec should be able to run them" then fine; but if su is expected to be fully-privileged, then pkexec should be too.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 26, 2014 23:41 UTC (Tue) by dowdle (subscriber, #659) [Link] (5 responses)

There has been some discussion the CentOS Development mailing list asking if their was interest in a grsecurity patched CentOS kernel. There was quite a bit.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 6:28 UTC (Wed) by Lionel_Debroux (subscriber, #30014) [Link] (4 responses)

I'd be all for it as well. The availability of pre-built packages for the main distros, and upstreaming some PaX / grsecurity bits in the first place, are probably the only ways to greatly expand the number of grsecurity users, and in turn improve security. Randomized struct layout would be pretty much unique to each build, and make it harder to leverage the "exploit works on all machines using distro kernel" effect.

But someone will have to pay for at least part of the cost of increased development and QA (on PaX / grsecurity, downstream distros, mainline Linux, upstream projects): testing, bug reporting, bug triaging, adding paxctl calls to the packaging files for the applications which will not work with some PaX features (e.g. most JITs fail with MPROTECT), fixing and improving the code of applications so that they do not trip PaX & grsecurity protections, etc.
Nobody paid for those costs before, but for reduced insecurity and improved security, it needs to be done some day. Much easier said than done...

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 27, 2014 23:29 UTC (Wed) by deepfire (guest, #26138) [Link] (2 responses)

> But someone will have to pay for at least part of the cost of increased
> development and QA (on PaX / grsecurity
...
> Nobody paid for those costs before

I suggest us to think about what it means..

Nobody of those in position are interested in making things significantly more secure?

Why is that?

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 31, 2014 9:29 UTC (Sun) by nix (subscriber, #2304) [Link]

Perhaps because their developer(s) make Ulrich Drepper look like the soul of reason and charm? It's the Schilling effect: be venomous enough and nobody will work with you voluntarily, no matter how important the job you do.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 31, 2014 12:40 UTC (Sun) by mpr22 (subscriber, #60784) [Link]

Because technical measures that provide enhanced security are not always a net gain even when the people who need to be able to trust a given system are the same as the people who own and control that system.

PaX and grsecurity have been noted, by their maintainers, to have performance penalties attached to them (IIRC the default configuration of grsecurity doesn't enable all of its security enhancements for reasons along the lines of "diminishing returns", but even the default configuration has a performance penalty attached). Performance reductions, for many significant users of the Linux kernel, translate directly into increased electricity and rackspace bills.

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 28, 2014 14:49 UTC (Thu) by thestinger (guest, #91827) [Link]

The PaX exceptions don't need to involve any effort on the part of other packagers. In Debian, they could be done with a single list of exceptions applied via dpkg hooks. I've taken a similar approach for the Arch Linux grsecurity support, because I couldn't get other developers interested. I think it's far easier like this anyway:

https://github.com/thestinger/paxd/blob/master/paxd.conf

The poisoned NUL byte, 2014 edition (Project Zero)

Posted Aug 29, 2014 18:08 UTC (Fri) by kleptog (subscriber, #1183) [Link]

Doesn't this kind of prove Linus' point that there's no point calling out bugs specially as security bugs and just assume everything is exploitable and needs to be fixed?


Copyright © 2014, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds