Of course this goes to a General Resolution
Of course this goes to a General Resolution
Posted Feb 12, 2014 4:35 UTC (Wed) by mbt (guest, #81044)In reply to: Of course this goes to a General Resolution by jmorris42
Parent article: The Debian technical committee vote concludes
1. Good software engineering puts a premium on clearly delineated modules having well documented interfaces. Modules should avoid having too close of an idea of the internals of other modules. This kind of clear separation serves both to make the whole more maintainable and also to facilitate testing. This is not a Unix-only characteristic! I have found this out in many years of being a programmer.
It would be a wonderful thing to be able to build a system with some of the modular components enabled and some disabled and, better still, be able to make some of the components stand alone where that is reasonable and possible. Yes, that would mean the possibility to package some of the components separately. That's part of the benefit of modularity.
The systemd approach, by contrast, is to put everything together into one whole. If you don't need all the things in it, too bad. One case in point: multiseat. This seems to have been one of the biggest drivers behind systemd and also Gnome's dependence on systemd, yet it is hard for me to imagine why such an esoteric usage case is even desirable, let alone have its complicated code and configuration being imposed on *every* system that uses systemd.
The Gentoo developers asked really nicely for a build option in systemd to allow building udev only; they even submitted patches. No. The package was not to be modularized that way. (If you want an in-the-flesh example of embracing and extending, you need look no further than udev.)
To the objection that the kernel, like systemd, is monolithic, let me observe that the kernel has *thousands* of configuration options. You can tune it to quite a degree; it can be really big or really small. There's a little bit of choice available in systemd, like whether to include an HTTP server (??!!!??), but for the most part, you get the whole thing.
2. How do you get to systemd? This process which is supposed to run at PID 1 has open sockets, sits on top of an IPC system, has logging and configuration files all over the place, handles device events, makes hooks for a GUI tool, and deals with a stream of login credentials (which come from outside the system, remember). This presents a *HUGE* attack surface. It's almost an engraved invitation to mischief. There is, unfortunately, a continual pattern of exploits being discovered in software packages. Nearly all of these packages are much more modest than systemd, and many are much older. How much will you like systemd when zero-day exploits start showing up?
3. One other word about modularity. Lots of big packages--Apache, MySQL, Postgres, KDE, Python, and PHP come right to mind--are cross-platform and have quite a number of build options. Many of these options are for the purpose of fitting into one environment or the other. Now there will have to be configurable adaptations in some of these packages to accomodate systemd. Why can't systemd have the same kind of configurability?
4. The third argument leads into the fourth. One reason to design a package with rather few build options is because there has not been enough developer time and user experience with it to make the options. Systemd development has gone at a frenetic pace, and even now it seems poised to continue to add to its feature set. To wit: it is growing; it is still not mature. I don't consider it stable enough to use.
Posted Feb 12, 2014 5:36 UTC (Wed)
by jmorris42 (guest, #2203)
[Link]
If mine are all invalid or easily addressed this issue is probably closed and thee and me[1] lost a fair fight. If I'm right the fun ain't even started in earnest yet. Debian has great processes to answer technical questions but questions without such an answer can get messy.
[1] More like thee lost since I'm only a fairly recent convert to Debian and not a developer.
Posted Feb 12, 2014 6:20 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (11 responses)
>It would be a wonderful thing to be able to build a system with some of the modular components enabled and some disabled and, better still, be able to make some of the components stand alone where that is reasonable and possible.
> The Gentoo developers asked really nicely for a build option in systemd to allow building udev only; they even submitted patches. No. The package was not to be modularized that way. (If you want an in-the-flesh example of embracing and extending, you need look no further than udev.)
>2. How do you get to systemd? This process which is supposed to run at PID 1 has open sockets, sits on top of an IPC system, has logging and configuration files all over the place, handles device events, makes hooks for a GUI tool, and deals with a stream of login credentials (which come from outside the system, remember). This presents a *HUGE* attack surface.
>4. The third argument leads into the fourth. One reason to design a package with rather few build options is because there has not been enough developer time and user experience with it to make the options.
Posted Feb 12, 2014 22:38 UTC (Wed)
by mathstuf (subscriber, #69389)
[Link] (1 responses)
Seeing as it is Gentoo and users tend to build from source, avoiding that cost if the end result wasn't wanted might be a goal. Maybe a "./configure; make udev; make udev-install" would work instead in the ebuild?
Posted Feb 13, 2014 0:57 UTC (Thu)
by ABCD (subscriber, #53650)
[Link]
Posted Feb 13, 2014 5:11 UTC (Thu)
by mbt (guest, #81044)
[Link] (8 responses)
Consider one package that certainly has the right to say that its API is in a really stable state: TeX. It's at version 3.14159265, a position to which it has taken more than 30 years to converge. TeX does just one thing--very, very well. It took more than 10 years to get into a reasonably fixed state. And that's from a guy who has a legendary concern with program correctness. Call me somewhat less impressed by systemd's stability guarantees.
Actually, the point I was raising was not about the external API, important as that is. I was looking at the major components (core process-starter/stopper, device-event handler, CGroups manager, logger, cron manager, login/session manager, power management, and others I have likely missed). Why are they not separable? I notice a FD.O has a page on minimal builds for systemd, but I have not found documentation on what you get by enabling or disabling some of the components and how well supported the resulting system would be.
> And again, you're in luck! Most of systemd dependencies are either trivial or can be turned off.
The build process to get a standalone udev is painful. It is meant to be an indivisible part of systemd, yet it is (still) separable. To build just udev you have to build all of the core systemd, and then separate out just the udev parts. This also means hoping that sometime in the future they don't make a hard dependency between the actual systemd daemon and the udev component. This kind of hard dependency has already befallen logind: since systemd-205 the Gentoo developers have been unable to make a standalone logind because logind is now inextricably bound with the cgroups controller. Did there really *have* to be such a tight binding? I absolutely do not think so. All the same, if you want logind, you must allow systemd to assimilate you.
Re. the attack surface that systemd's many interfaces present:
> Nope. Keeping the set of variants to the barest possible minimum is good. For example, Linus had resisted forking Linux into a 'server' and 'desktop' variants - with great results.
Posted Feb 13, 2014 6:05 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
> Consider one package that certainly has the right to say that its API is in a really stable state: TeX.
I wanted recently to render my thesis written in 2000 in TeX. Turned out that the recent MikTex no longer can compile it. And I had to go and fix some of the modules used.
Oh, and all of the text is written in KOI-8R encoding and it's not that trivial to make some of the new editors to switch text encodings.
> Actually, the point I was raising was not about the external API, important as that is. I was looking at the major components (core process-starter/stopper, device-event handler, CGroups manager, logger, cron manager, login/session manager, power management, and others I have likely missed). Why are they not separable?
> I notice the Gentoo ebuilds for systemd don't have USE flags to control whether to build components like logind. That is very telling since Gentoo loves configurability. How well does systemd work with some of its "optional" components missing? It would be nice to have some clarity.
> In current systems, an attacker might never be sure of the mix of components. Given the variety of init systems, loggers, cron daemons, power-control systems, and such in current Linux installations, a one-size-fits-all attack is harder to mount.
>Systemd, by contrast, stands for a monoculture of system components. Biology tells us a lot about the dangers of monocultures.
> Linux manages a truly impressive thing. A single codebase works for generating kernels from the whole gamut from watches to supercomputers. There are no forks, but there are more than 3000 configuration options.
Posted Feb 13, 2014 7:52 UTC (Thu)
by mathstuf (subscriber, #69389)
[Link]
We all run a single kernel codebase and I'm sure spender or PaXTeam could tell you all about the problems you have running it before you stack stuff on top of it.
And besides, are you really arguing that having a pile of stuff that attackers can't deal with it anyways? Have you seen the stuff that gets worked around by people? Do you really expect every sysadmin to require such a diverse skillset just to deal with whatever heterogeneity in their system set may be?
For logind and cgroups, logind uses a DBus API to talk to the single cgroup manager. What else can it do in the New World Order of cgroups? If you implement that API (like, say, systemd-shim), you can get away with not using systemd as PID 1.
Posted Feb 14, 2014 19:50 UTC (Fri)
by smurf (subscriber, #17840)
[Link] (5 responses)
If you want to stop processes you need to be PID 1, otherwise you get a race condition.
> device-event handler,
Starting processes depends on events.
> CGroups manager,
When you start a server you need to put it in a CGroup.
> logger,
On-disk logging is separate. In-memory logging cannot be, since you want to start logging before root is even mounted.
> cron manager,
Because cron jobs don't just depend on the time of day, these days. They depend on being on AC power, or on the database running, or whatever. So best handled within the
> login/session manager,
separate.
> power management
For instance, shutting down? init needs to cleanly stop all its children and then exec something that's in RAM, otherwise the root file system cannot be unmounted cleanly.
You seem to forget that all these jobs need to closely communicate with each other, need to run all the time, need to serialize their internal state if they're to be updated without rebooting (i.e. you cannot just kill and restart your event handler and expect that everything is magically still hunky-dory.
So you either deal with five processes and their communication overhead and the subtle race conditions which are *certain* to crop in … or you put all of this into one single-threaded program which always has consistent local state which you can serialize safely if you want to re-exec yourself after an upgrade … and which you call "pid 1", necessarily.
In summary, can we please stop pretending that Lennart & Co. (a) put all of this into one huge binary that/s /sbin/systemd, (b) cobbled the part that _is_ PID 1 together out of spite, or because they didn't know better? 'Cause, you know, it's quite obvious that they actually thought about this and had/have sound technical reasons.
Posted Feb 18, 2014 17:23 UTC (Tue)
by HelloWorld (guest, #56129)
[Link] (4 responses)
Posted Feb 18, 2014 18:44 UTC (Tue)
by smurf (subscriber, #17840)
[Link] (3 responses)
Anyway, the point isn't whether it's possible to run systemd as PID-2. You'd still need a way to signal PID-1 that it should please pivot its root file system to the RAM disk and exec /shutdown, so that your root file system can be unmounted cleanly. And probably some other minor quibbles that seem perfectly solvable, but also completely unnecessary when you can just have the features in PID-1.
Which begs the question: why bother? I still haven't seen any reason what the advantage of running systemd as pid-2 (or pid-2+pid-3+pid-4) would be. "Clean separation of responsibilities into separate processes" is not an argument I can accept, because they all need to run continuously and they all need to talk to each other, so you get increased complexity for no net gain.
Posted Feb 19, 2014 10:15 UTC (Wed)
by HelloWorld (guest, #56129)
[Link] (2 responses)
> Which begs the question: why bother? I still haven't seen any reason what the advantage of running systemd as pid-2 (or pid-2+pid-3+pid-4) would be.
Posted Feb 19, 2014 11:06 UTC (Wed)
by mchapman (subscriber, #66589)
[Link]
I think the alternative complicates things.
If systemd running as PID 2 and marked as a child subreaper were to crash, then its children would be inherited by PID 1. Even if PID 1 were to restart systemd, the new systemd wouldn't be able wait on those reparented processes any more. PID 1 would be responsible for reaping them when they exit, and PID 1 would need to pass on notifications to that effect to the systemd process (so that it could re-exec them or whatever).
In short, I think using a separate child subreaper brings as many problems as it solves.
Posted Feb 19, 2014 11:34 UTC (Wed)
by smurf (subscriber, #17840)
[Link]
*Every* program which holds a file open on the root file system (or any file system, for that matter) needs to either exit, or exec() a program within the new (RAM disk) root. Otherwise you cannot unmount the root FS.
PID-1 may not exit. Therefore it's its job to exec the last step. PID-2 cannot do that. (OK, it could call the unmount-and-reboot program on the RAM disk after triggering PID-1 to exec a new init there, but again: what would be the point of that additional complexity?)
Posted Feb 12, 2014 6:21 UTC (Wed)
by josh (subscriber, #17465)
[Link]
Almost none of the things you described live in PID 1. udev, logind, and journald all live in separate processes. And as for "login credentials", systemd actually goes to great pains to never invoke NSS from PID 1.
Posted Feb 12, 2014 12:01 UTC (Wed)
by joyuh (guest, #95216)
[Link]
Use VMs or seccomp if you want to run code with reduced privileges.
Posted Feb 12, 2014 20:45 UTC (Wed)
by Wol (subscriber, #4433)
[Link]
Let me explain why it is desirable. You have a powerful computer at home (mine is 16Gb ram, X3 processor, 1Tb disk).
My wife objects to computers everywhere (and we don't have much space for them, anyway).
At which point I start cursing because she's hogging the computer!
With multiseat I can just use the same computer (and I've tried connecting over X - it is *appallingly* laggy :-(
For those people who grew up with multiseat (ie started on minis or mainframes), it just seems the natural way to do things - and with modern powerful PCs, it seems even more natural!
Fortunately I've got a spare computer I can switch over to systemd to test out multiseat before I send it live on our main system.
Cheers,
Of course this goes to a General Resolution
Of course this goes to a General Resolution
Then you're in luck! Systemd has clearly defined external interfaces with a stability guarantee.
And again, you're in luck! Most of systemd dependencies are either trivial or can be turned off.
They affected only the build system, not the end result.
And so does the variety of tools that are used for all this stuff right now.
Nope. Keeping the set of variants to the barest possible minimum is good. For example, Linus had resisted forking Linux into a 'server' and 'desktop' variants - with great results.
Of course this goes to a General Resolution
That is effectively what the ebuild does, but there are a lot more targets required for both the make udev and make udev-install type targets, as well as needing to pass the proper arguments to ./configure, some of which are to try to disable dependencies that upstream considers mandatory (because they are needed for systemd itself, but not for systemd-udevd).
Of course this goes to a General Resolution
Of course this goes to a General Resolution
> Then you're in luck! Systemd has clearly defined external interfaces with a stability guarantee.
Stability is a good thing. What is not so fun is the great deal of disruption required to get to stability. Besides that, I'm really surprised that the systemd maintainers can be so sure that they've reached that point. It is certainly not to say that they've put in a huge amount of effort. I'm just saying that's a *lot* of public interface to be sure that you've got it set in stone.
I notice the Gentoo ebuilds for systemd don't have USE flags to control whether to build components like logind. That is very telling since Gentoo loves configurability. How well does systemd work with some of its "optional" components missing? It would be nice to have some clarity.
> And so does the variety of tools that are used for all this stuff right now.
In current systems, an attacker might never be sure of the mix of components. Given the variety of init systems, loggers, cron daemons, power-control systems, and such in current Linux installations, a one-size-fits-all attack is harder to mount. Systemd, by contrast, stands for a monoculture of system components. Biology tells us a lot about the dangers of monocultures.
Linux manages a truly impressive thing. A single codebase works for generating kernels from the whole gamut from watches to supercomputers. There are no forks, but there are more than 3000 configuration options. The range of things you can get out of the kernel sources are well supported. I stand in awe of those developers.
Of course this goes to a General Resolution
Yup. And they're committed to supporting it. And we have assurances of that from RedHat.
Funny you mentioned it. It's a great example. The core is indeed very simple and robust, but modules that are required to do anything real are most certainly not.
Why should they be? BTW, power manager is separate.
Perfectly fine. I'm using it with only journald (and udev, of course) on embedded devices. I even removed localed and other auxilaries.
You need only one chink in the armor to get in. Are you sure that your syslog daemon is not subverted by NSA?
Computing is not biology. And I'm a computational biologist.
Yet the process scheduler has only single implementation.
Of course this goes to a General Resolution
Of course this goes to a General Resolution
Of course this goes to a General Resolution
You mean because orphans are reparented to init? That can be avoided with prctl(PR_SET_CHILD_SUBREAPER). So I don't really see why systemd needs to run as PID 1 instead of, say, 2 nowadays.
Of course this goes to a General Resolution
Of course this goes to a General Resolution
Why is it PID 1 who needs to do that?
The kernel will panic if PID 1 crashes, so it should be as simple as possible. Now, systemd never actually crashed on any of my systems, but why take the risk if you don't have to?
Of course this goes to a General Resolution
Of course this goes to a General Resolution
Of course this goes to a General Resolution
Of course this goes to a General Resolution
Of course this goes to a General Resolution
Wol