LWN: Comments on "Systemd and containers"

This model of containerization is a grand error

anselm — Sun, 05 Jul 2015 14:23:28 +0000

Who'd've thunk German was without consistent transcription rules.

What, like English?

In German you can generally spell “ö” as “oe” if you don't have umlauts. But that doesn't mean every “oe” you encounter was originally an “ö”, or that you are always free to do the reverse replacement, especially in the names of people or places. For example, you would never get away with “Göthe” in place of “Goethe” (Germany's most famous poet) outside of the realm of lowbrow comedy. Transcription rules don't really enter into this.

This model of containerization is a grand error

viro — Sun, 05 Jul 2015 03:39:22 +0000

Oh, for pity sake! I loathe the guy, but this is ridiculous. Names are arbitrary tags. Etymology doesn't matter - it's not as if they had any meaning other than identifying the object they apply to. He gets to choose the orthography; insisting upon a different spelling simply demonstrates that the person insisting is a pretentious wanker.

*plonk*

This model of containerization is a grand error

ksandstr — Sat, 04 Jul 2015 23:46:43 +0000

>No. Lennart never spells it with 'ö' which makes it wrong.

Interestingly the Wikipedia article on German orthography agrees with this reading. It appears you'll have to read it as a disrespectful stab up 'til now, and that I'll be avoiding the name altogether in the future. Call it going from petulance to erasure.

Who'd've thunk German was without consistent transcription rules.

This model of containerization is a grand error

mathstuf — Sat, 04 Jul 2015 19:47:05 +0000

> But it does.

No. Lennart never spells it with 'ö' which makes it wrong. I'm sure my last name had an umlaut at some point too, but I'd still spell it 'oe' anyways today. It isn't used as such in any official documentation or by me, so it is wrong to use it. That is the case here too (AFAIK).

This model of containerization is a grand error

ksandstr — Sat, 04 Jul 2015 15:17:33 +0000

Quoth the Wikipedia:

Lennart Poettering (born October 15, 1980) is a German computer free software programmer [...]

Whatever your implied argument on his current nationality, the origins of his name are German, from which its spelling and pronounciation follow. That is the point under argument.

This model of containerization is a grand error

ksandstr — Sat, 04 Jul 2015 15:13:44 +0000

On the contrary, the question is not what I put in a container, or what the GNU/Linux distribution puts in one. Rather, containers used in this fashion create an interface for distribution-independent blobs from (tautology aside) outside any given distribution and its anti-bundling policies. Such blobs will invariably be bundled with their dependencies: how could it even be otherwise given filesystem separation to the degree of "soft VM"?

In the absence of standardization for intra-container automation (as stated in the "PM must reach" paragraph), this leaves users of security-broken web-download software with two workable options: either they wait for the container publisher's update (which may come in the form of a major version update, perhaps for a cost), or they build the fixed container themselves. Obstacles to the latter can be many, such as the recent Firefox bundling of in-browser DRM plugins; and alternatives to these two all boil down to bending over for the NSA.

The effect of containerzation in the manner proposed in the main article is that Debian, Mint, Gentoo, etc. users will be as ripe a target market for Linux desktop app stores as Red Hat's customer base is for its RPM repository. Iterated, this development converges in the death of the individual distribution in favour of an effective monoculture as preferred by proprietary software companies. With it comes the death of Free Software.

In closing, systemd must be destroyed.

This model of containerization is a grand error

ksandstr — Sat, 04 Jul 2015 14:48:31 +0000

How about instead of this generally passive-aggressive mumbling, you say how you really feel? You could even go as far as to refute my points directly.

Failing that, you could niggle at my insubtle hypothesis that eventually Pöttering will full-bodily enter his own fundament and thereby vanish. Or you could take issue with the hyperbole referencing Brave New World, a book concerning dystopian conformity; or the company store, a symbol of dire working-class exploitation in pre-New Deal America. Certainly there are individual phrases or words that could similarly serve as points of deflection, such as the millennia-old "foo must be destroyed" statement of creed[0], or my framing the putative motives of systemd "fanbois"[1] as motivated by fashions (i.e. positive peer pressure) alone.

That's assuming that the capacity for such refutation and/or secondary niggling beyond a variant spelling exists, that is.

Until then, I wish you a good day.

[0] entirely pre-empting the "dog whistle" shibboleth argument, in case this went by under the radar
[1] that's class right thur

This model of containerization is a grand error

Cyberax — Sat, 04 Jul 2015 14:39:29 +0000

Lennart Poettering is not from Germany.

This model of containerization is a grand error

ksandstr — Sat, 04 Jul 2015 14:26:46 +0000

But it does. In German the umlaut is regularly transcribed as an "oe" (or "ae" for ä). To contrast, in Swedish an "ö" would be ASCIIfied by stripping the diacritic altogether, which is common in e.g. E-mail. I'm using the non-canonical spelling because my keyboard has that letter, because "oe"'s pronounciation is that of "ö", and to avoid bolstering the marketing effect of someone's name starting with "poet"-- because who knows, perhaps Americans are genuinely that dumb. On the other hand Pöttering's first name is already a term of abuse, so there's good reason to avoid that as well (beyond unwarranted familiarity).

As for these other pop-psychology Internet shrink hypotheses about "asserting dominance", piffle! We are not cats and dogs.

This model of containerization is a grand error

dlang — Wed, 24 Jun 2015 04:06:26 +0000

> And in a true UNIX lore fashion, no software but one should deal with SSL, since this otherwise would violate the idea of doing one thing and doing it right.

you completely misunderstand the Unix "do one thing and do it well" mantra. That doesn't in any way prohibit you from having multiple things that do the one job, it just means that one tool shouldn't try to do lots of different jobs.

This model of containerization is a grand error

misc — Sun, 21 Jun 2015 11:48:48 +0000

In the end, it all depend on how and what you put in the containers. Nothing would prevent automation of containers rebuilding if you assemble them from the existing distribution infrastructure. The problem is one of policy, not of technology, since tar.gz, static compiling, source code bundling or rpm/deb/whatever all suffer from the risk of having a component not cleanly separated. And that's not theorical, since that's why Fedora mandate to not bundle anything ( unless exceptions ). Debian do also have the same policy, following a zlib problem back in the days.

On the desktop side, getting application containers from Fedora rawhide running on a centos 6 might solve the issue of people wanting latest version of something without upgrading to rawhide. It doesn't solve every problem, likely bring some news, but that's worth testing and doing. Companies handling lots of traffic ( facebook, google, twitter among others ) have been using that since years on server side, so I think they would have noticed security issues.

And in a true UNIX lore fashion, no software but one should deal with SSL, since this otherwise would violate the idea of doing one thing and doing it right. So application containers push for more of the UNIX paradigm on the server side.

This model of containerization is a grand error

misc — Sun, 21 Jun 2015 11:22:34 +0000

I agree with your point, I would go as far as saying that it can extend to more than persons ( example, a company in Seattle ).
Or do we just personify the said companies and therefore treat then as people, still not sure on that one.

Systemd and containers

anselm — Sat, 20 Jun 2015 14:19:18 +0000

Go back under your bridge, troll. This has been debunked so many times it is no longer funny.

Systemd and containers

dakas — Sat, 20 Jun 2015 14:18:03 +0000

anyway is good to have a choice

The systemd universe is about not having choices. It's all or nothing.

Systemd and containers

flussence — Fri, 19 Jun 2015 22:05:17 +0000

> pixman and glamor will keep older fixed function graphics adapters reasonably operational, all the way back to the good ol' ATI mach64 and matrox G200w

If they get glamor's GLSL shader engine to work on those OpenGL 1.4 chips, when even my GL2.1-capable i945 refuses to start X with it enabled, I'll be impressed. Good luck with that.

This model of containerization is a grand error

raven667 — Fri, 19 Jun 2015 17:13:38 +0000

It usually isn't effective in creating submissiveness on the person its being said to, it's more about the ego of the person doing the saying, demonstrating that what they say is under their own control and they can be rude with little consequence. That is something which can puff up a weak ego, if only slightly.

This model of containerization is a grand error

anselm — Fri, 19 Jun 2015 08:52:37 +0000

I think that in order to “assert dominance” over Lennart Poettering you would have to do a bit more than mis-spell his name.

This model of containerization is a grand error

raven667 — Fri, 19 Jun 2015 00:59:35 +0000

You are right, its a dog whistle to make your feelings clear to like minded people and the practice of giving nicknames can be a way to assert dominance over someone by controlling something so fundamental as the persons name.

This model of containerization is a grand error

Cyberax — Thu, 18 Jun 2015 18:07:25 +0000

It's a shibboleth to mark anti-Lennart fanbois.

This model of containerization is a grand error

peter-b — Thu, 18 Jun 2015 15:46:22 +0000

Poettering doesn't contain a "ö".

This model of containerization is a grand error

ksandstr — Thu, 18 Jun 2015 14:45:24 +0000

Long story short, installing applications as self-contained mega-blobs which get deduplicated by filesystem magic is a bad idea because it makes security holes (e.g. Heartbleed, GRIME) persist per blob. To contrast, in traditional Unix-alikes those OpenSSL bugs were patched per OS image because OpenSSL is a shared library; and only binaries that linked it statically required dedicated upgrading. For the tin-foil types in audience, this is the ideal situation for the NSA and other cybercrooks: in the brave new Pöttering world, exploitable holes persist beyond automatic package management's capacity to repair.

To achieve parity with regard to ease of maintenance, package management must reach into each container, perhaps with a heavy layer of metadata in each; or PM must become per-containee with an equivalent high-level interface to the container. Shortly after implementing this[0] Pöttering will finish (topographically speaking) swallowing himself like a German ouroboros, and disappear without a trace.

The purpose of this overall desk-top inner-system exercise has gone underdiscussed, beyond it being the fashions ("all the rage") at the moment. My hypothesis is that Red Hat is still entertaining the idea that all of GNU/Linux should be reworked into an App Store[1] platform: the opportunities for increased control, parlayed into cashflow, are currently going unexploited outside of the respective installed bases of Apple, Microsoft, and Google. Fundamentally, such development is undesirable for everyone because it makes the users[2] of software less capable of modifying mega-blobs they've received from the Company Store, and operating their modified versions transparently; this makes the stated current direction of systemd inimical to the Four Freedoms.

In closing, systemd must be destroyed.

[0] or (more likely) some typically overengineered facsimile for novelty's sake
[1] and like, that shark got jumped over before "let's have an app store for home automation widgets", i.e. sometime in the late aughties at the latest
[2] such as you!

Systemd and containers

anselm — Thu, 18 Jun 2015 09:08:30 +0000

There are lots of systems that profit from being based on systemd but have no conceivable need for NetworkManager (mostly because NetworkManager's strength is dealing with a changing network environment, while these systems may be sitting firmly on a desk or in a rack somewhere). It makes sense to investigate whether such systems could use something simpler than NetworkManager+dnsmasq.

Systemd and containers

mchapman — Thu, 18 Jun 2015 08:38:34 +0000

> I have seen code which switches root to the initramfs and runs a binary from there on shutdown. Wouldn't that mean that the initrd needs to be kept resident all the time?

No, the initramfs can be repopulated upon shutdown.

For example, with Dracut this is done by dracut-shutdown.service, which is pulled in by systemd's shutdown.target unit. See https://www.kernel.org/pub/linux/utils/boot/dracut/dracut... for details.

Systemd and containers

dakas — Thu, 18 Jun 2015 08:07:58 +0000

NetworkManager is a useful but quite complicated piece of software,

So does systemd differ by being not useful or by being not complicated? I don't quite see the point you try to make against this being NIH syndrome.

Systemd and containers

cesarb — Thu, 18 Jun 2015 00:24:51 +0000

> There is a limit: initial ramdisks are not swappable (unlike, say, tmpfs). (This is why they are generally cleaned out and thrown away when switching to the initial root, so as not to waste precious physical RAM).

I have seen code which switches root to the initramfs and runs a binary from there on shutdown. Wouldn't that mean that the initrd needs to be kept resident all the time?

Systemd and containers

nix — Wed, 17 Jun 2015 12:26:12 +0000

Why make an arbitrary limit on the ramdisk to exclude it from being used as a rescue system?

There is a limit: initial ramdisks are not swappable (unlike, say, tmpfs). (This is why they are generally cleaned out and thrown away when switching to the initial root, so as not to waste precious physical RAM).

However, on a rescue disk it's not like you can rely on swap anyway, and on modern systems this is not much of a limit: I've seen ramdisks that contain entire compiler toolchains and even kernel source trees for building needed kernel modules on command. You wouldn't want to keep that lying around in RAM on a general-purpose system (can you imagine how big the initrd/linked in initramfs must be?), but in a rescue disk it's fine.

Systemd and containers

raven667 — Tue, 16 Jun 2015 17:16:43 +0000

> Sane size initial ram discs just don't and can't provide that but small root filesystems with /bin and /lib can.

Why make an arbitrary limit on the ramdisk to exclude it from being used as a rescue system? The initial ramdisk/ramfs mechanism is robust enough that it can load an entire OS installer/upgrader image. It seems that you've created a tautology by redefining sanity as not having a useful ramdisk to prove that ramdisks aren't useful.

> fact I am neo-luddite

Well, OK then, you can use what ever system you like but when building a system to distribute to others, that has to be flexible and robust enough to handle whatever crazy new thing they try to do with the system, features like a complete initial ramdisk able to mount anything, are kind of a requirement and can't just be defined away.

Sure, if you only ever had to support one use and one hardware then you could remove a ton of abstractions and have a greatly simplified system but that's not something you can do if you are distributing systems for others to use in any environment.

> If everything was in /usr then I could not boot, period: systemd dies and I have to mount /usr and /var manually for it.

You pretty much have to have all filesystems mounted before you exit the initial ramdisk and start services in the live root filesystem, if you try to chroot into a system which is half missing then it won't work.

> Initial ram discs which duplicate tools that are installed on a box, and are hard to update, *are* kludgy hacks.

Initial ram disks provide a mechanism for the tools on your system to be accessible with a minimum of fuss because they are loaded by the firmware so that your filesystem is fully ready to go when you pivot into it and you aren't trying to run on a half-borken system. There is a policy in the kernel of pushing setup into user space of anything which can be done by userspace so you need to have a userspace capable of setting everything up, loading modules, mounting filesystems, iteratively as busses come online exposing new hardware, before you can transition into your root filesystem.

Systemd and containers

dowdle — Mon, 15 Jun 2015 21:11:48 +0000

Here's a video of the same talk (pretty much) from an earlier conference (April 2015):
https://www.youtube.com/watch?v=d4SwL2t5Yh4

Systemd and containers

dps — Mon, 15 Jun 2015 14:03:36 +0000

Anybody that has had significant hard disc problems and other complex problems knows that a functional system that does not require a large filesystem, like /usr, to work is worth a lot when the chips are down. Sane size initial ram discs just don't and can't provide that but small root filesystems with /bin and /lib can. The tools you need to fix your initial ram disc are unlikely be available if that is all you have.

If I want an aggressively minimised box then I just don't use systemd: kernel modules, dbus, udev and network manager are all bloat which stripped down firewall machine with fixed hardware does not need and therefore should have. Anything that requires them should be rejected when there is a reasonable alternative that does not. I might actually want some of the features on offer on a desktop but have yet to find any use for dbus whatsoever. It is possible that the fact I am neo-luddite and don't use linkedin, facebook, twitter, GUI file system interfaces, cloud services, etc is a sufficient explanation for that.

I also find systemd has significant problems when /var is a separate filesystem because some things it puts in /var suffer an existence failure when that filesystem is mounted. systemd militates against having /usr on a separate partition which is mounted read only, making the desired separation of fixed and mutable data harder. If everything was in /usr then I could not boot, period: systemd dies and I have to mount /usr and /var manually for it.

Initial ram discs which duplicate tools that are installed on a box, and are hard to update, *are* kludgy hacks.

Systemd and containers

alan — Mon, 15 Jun 2015 09:58:25 +0000

Instead of implementing a dhcp server, was any effort made to reach out to dnsmasq developers to add the additional support needed? If not then this is definitely NIH syndrome, reimplementing a service that gives you 95% of what you need without investigating what is necessary to get support for that additional 5%.

Systemd and containers

grawity — Sat, 13 Jun 2015 13:34:18 +0000

I remember spending some time with that. Both systemd's libnss_resolve, and (I think) Avahi's libnss_mdns, do return the scope index. Unfortunately, glibc throws it away.

(libnss_dns of course doesn't return the scope because it doesn't know the scope.)

Systemd and containers

dlang — Sat, 13 Jun 2015 02:09:58 +0000

It's ironic to hear systemd folks say that new standards should not be created where there are existing ones. They have seemed to be the kings on NIH and creating new 'standard' ways of doing things.

I strongly disagree that every daemon that runs on the host should run in every container as well. That leads to a large amount of unneeded duplication, and probably some security issues over time as well. If the software running in the container doesn't need functionality, it shouldn't be there. For some daemons, it's all or nothing, if they need even a single feature they need the whole thing, but for many that's not the case and you can eliminate the vast majority of the features/complexity/potential bugs from inside the container without hurting the container functionality.

Systemd and containers

Kamilion — Fri, 12 Jun 2015 09:35:22 +0000

... Nneeeggh! I don't really want all those directories cluttering up my /.

I don't want /bin, I don't want /sbin, I don't want /lib, I don't want /share, I don't want /local, and I certainly don't want /include.

It's kind of a shame that sysfs took over /sys otherwise I'd really push for just changing /usr's name to /sys.
But heck, people can't even agree on what /usr means today -- I keep getting told it's Unix System Resources, but to me it's always been the /usEr/ directory.

"I do not at all have the mind of a bully... in my mind bullies are intolerant of contrary opinion, domineering and rather cowardly. I would hope that none of those terms could be fairly used in describing me." -- Conrad Black

But that would be one of those changes that breaks the world because too many things rely on /usr/bin/<something>.

Thankfully on a lot of modern systems, /bin, /sbin, and /lib are just links to their /usr counterparts. One day they'll dissipate.

Eventually the old ash/bash/perl scripts will get replaced by python3, rust, and go by the distros. Maybe even Swift if it really takes off. I'm kind of skeptical about that one. Coreutils installations will start to be rare as whatever default dynamic language on the installation medium (python3) handles doing all the unixy regexes and transforms without shelling out of process...

X will go back to being an immediate-mode drawing protocol and DRI will be a laughable thing of the past once the wayland ecosystem solidifies. Most of the application toolkits have already supported it for quite a while; then xwayland will handle loading the required xorg components when necessary, like when I want to play a good old fashioned game of xevil. Once enough time passes, xorg won't even be a default installed component anymore and it'll be like "whaaat, I have to go get xorg to run this? Pffft! I'm not going through that hassle, googleing for a native rebuild..." kinda like xQuartz on a mac...

pixman and glamor will keep older fixed function graphics adapters reasonably operational, all the way back to the good ol' ATI mach64 and matrox G200w (Darn you, integrated server KVM devices!) and various framebuffer devices, including spitting SPI out to a 320x240 touchscreen on a Pi's GPIO header.

"I always keep a firewall between my own travails and my perception of public-policy issues; otherwise I would retain no credibility as a commentator." -- Conrad Black

The internet is changing. Unix is changing. If you're not cool with that; Slackware and Linux From Scratch can always use the help! (Although LFS has a systemd version which may become standard...)

Lennart's swiped the three most useful apple services, mdns/bonjour/avahi, coreaudio/pulseaudio, launchd/systemd, ripped them to chunks, and had a go at putting together a lego tower with the results.

"There are some people whose opinion I value and respect and it would be very bothersome if I forfeited their respect. But the general public? I'm not preoccupied with the opinions of others." -- Conrad Black

I've played with that lego tower, and honestly, I like it a lot better than the old duplo sysvinit set.
Some of it gets a little strange, like finding a block of lego technics pieces in with your classics.

"Containers in my init system? It's more common than you think." -- Adapted internet meme

Ubuntu's wiki actually put together a great little mapping page between the upstart features and the systemd features:
https://wiki.ubuntu.com/SystemdForUpstartUsers
It's short, it's sweet, and it's mostly useful for users of other distros as well just to get a quick cheat sheet on systemd.

Systemd and containers

smcv — Fri, 12 Jun 2015 08:19:08 +0000

Recent proponents of merging the root filesystem with /usr initially thought this too, but came to realise that unifying into /usr made more sense than unifying into / - the lib, libexec, share, bin, sbin directories have more in common with each other than with anything else that has traditionally been at the top level.

/usr is not meant to change, ever, except during OS upgrades/package installations (and perhaps installations to /usr/local if you use that); so it can normally be mounted read-only, and if you're using something like OSTree, all of /usr should change atomically as a single unit. The "merged /usr" model does support a separate /usr filesystem, if the initramfs mounts it.

https://wiki.freedesktop.org/www/Software/systemd/TheCase... mentions some other possibilities, like a shared /usr between machines that have the same architecture and package-set (admittedly this makes more sense on OSs like Fedora, where services are normally installed but disabled by default, than on Debian derivatives, where unwanted services are normally not installed).

Systemd and containers

anselm — Fri, 12 Jun 2015 08:16:36 +0000

NetworkManager is a useful but quite complicated piece of software, and adding potentially several instances of dnsmasq into the mix isn't going to make things more straightforward. It would be useful to be able to deal with as many of the standard use cases as is reasonable with systemd-networkd (and if it plays well with containers, so much the better) and reserve the extra complexity of NetworkManager for those situations where it is actually needed (like dealing with lots of different wireless networks).

Systemd and containers

anselm — Fri, 12 Jun 2015 08:10:45 +0000

You only need to mount one file system to get the whole kit and caboodle (which is convenient for containers or remote storage) and your root directory isn't “polluted” by lots of names.

Systemd and containers

bandrami — Fri, 12 Jun 2015 04:58:48 +0000

So what does having /usr to begin with gain you in that scenario? Why not just put everything in /bin and /lib (and, hell, /include, /share, and /local if needed) and just skip out on /usr entirely?

Systemd and containers

cuviper — Thu, 11 Jun 2015 22:45:30 +0000

Yeah, it's not something I'd want to configure manually, but NetworkManager does it for me.

Systemd and containers

rahvin — Thu, 11 Jun 2015 22:42:42 +0000

The method you describe would require that you configure every time you change connection what connections go over what links. This can become exceedingly micromanaging and drive someone nuts. The systemd method ensures that you always get the results regardless of what links are up or active. As long as names don't overlap that is, if you have overlaps in names the systemd method would be a resolv nightmare.

Systemd and containers

luto — Thu, 11 Jun 2015 21:44:40 +0000

But dnsmasq would allow *.local (or whatever) to be forwarded to an LLMNR resolver.

There's one problem: for link-local applications, client software (especially on IPv6) needs an ifindex or other scope indicator. We don't have that using just resolv.conf-based mechanisms. I think that should be added independently of systemd-resolved, though.