|
|
Subscribe / Log in / New account

How did this guy test systemd exactly?

How did this guy test systemd exactly?

Posted Oct 10, 2010 22:59 UTC (Sun) by paulj (subscriber, #341)
In reply to: How did this guy test systemd exactly? by mezcalero
Parent article: Jaeger: systemd - and #osc2010

FWIW, I don't think it helps to foster the notion that anyone who has any opinion about systemd which doesn't praise it to high heaven must of course have not read the documentation. I have read the blog article several times, good chunks of the comments and some of the documentation posted since (the part you pointed me at is pretty much direct from your blog article). I don't think this helps encourage productive discussions.

I don't think I'm completely ignorant about what systemd is about:

1. Centralise the code for daemon creation and process management, and try get rid of things that do this themselves - often badly.

2. Have systemd to act as a socket proxy, auto-activating the services as required (ala ye olde Unix inetd, or Java rmid, etc). This allows many dependencies to be automatically worked out (static dependencies are made "redundant" is the claim), and so allow parallel launching. Activation is event-driven (see below).

I just noticed that the very latest DBus 1.4.0 now has support for handing off any spawning of services down to systemd - which is nice and addresses some of my concerns about the overlap between the two.

Your objection to events in upstart is that dependencies are propagated as a number of different "events", and it doesn't work. Systemd doesn't use this model for dependencies, however I presume you would still agree activation in systemd is event-driven (i.e. on the event of activity on the proxied fd). Reading service state and subscribing to events for changes in that state is also you wouldn't disagree with (provided you can read the current state and subscribe to new events in a race free manner). The latter is generally what I mean by event-driven, including the possibility where there is no state.

My concerns:

On 1: It's great to factor out and centralise code, but I'm not sure systemd will always be able to handle *all* process management directly. There's all kinds of weird cases people will want to handle, there are all manner of userspace service management thingies out there, from the more obvious ones (*inetd, rmid) to all kinds of custom ones. You can try generalise and handle most of them, but likely at some cost in simplicity (internally and admin visible - e.g. already, the docs for systemd service files contain an abundance of options), but it seems infeasible for systemd to be able to anticipate all possible demands of systems that want to spawn processes. Perhaps it'd be nice to have this stuff in a system *library*, useful to both systemd and other processes? E.g. systemd now supports DBus, but what about, say, Java rmid, which needs to be runnable as, and to configure new RMI services (for which it forks a separate JVM), as a non-privileged process; what about end-user DBus buses spawning services?

On 2: I'm not sure this is right. While there may be a number of services where it happens to work out, in the general case many dependencies are not visible*. That systemd has acquired various ways to manually specify ordering and activation is proof of this. Obviously there are lots of events that require richer systems to deal with, e.g. by modifying applications to publish state and reacting intelligently to changes in other applications' state (with some suitable IPC like DBus in there to glue it together). Essentially, I think even with systemd, you're going to still have a mass of files that have to describe properties of and relationships between services. That's info that's easy to get wrong, and otherwise can drift and get out of sync.

To summarise:

- I'm not sure it's healthy to centralise service management in 1 process. I suspect there'll always be need for additional specialised, sub-system centric process managers. (One may disagree on "need", but others will perceive such needs, and hence such will exist). So why not centralise the code in a library, so it can be used by /all/ process managers on the system?

- I can see how socket proxying is attractive, as it's a relatively easy change, but how much encoding of client properties can it really avoid? The only way to truly get dependency free bootup surely requires updating the applications to behave better?


to post comments

How did this guy test systemd exactly?

Posted Oct 11, 2010 14:48 UTC (Mon) by mezcalero (subscriber, #45103) [Link] (12 responses)

Yes, we try to centralizing process babysitting a bit. We want to cover all important system-level triggers and a few others. However, we see little need in porting everything that exists on this planet to use systemd for process babysitting. Different systems have different needs. We focus only on system and session services and their needs.

The word "socket proxy" is misleading, as it suggests that systemd was actually proxying every single byte from the network to the actual applications. This is not what happens. systemd hands off the real socket fd to the spawned applications and thus during runtime the performance is identical to services that called socket() themselves.

My comments regarding "events" in systemd and upstart refer to the fact that in upstart an "event" is a first-level object that is at the core on what it does and which is what the admin works with. While in systemd we internally use an event-based API such as epoll(), this is hidden to the outside and our first level objects are our "Units" and the dependencies between them.

I think there's little point in creating a library for spawning processes as you need a (system-wide) runtime component for a lot of the stuff that a good baby-sitter does, such as collecting info about crashes and suchlike. Also, it is essential that services are detached from the code that actually spawned them. That makes it important to fork them off some central process, so that they inherit a clean execution environment. Due to that you absolutely *want* this to be forked off PID 1.

And sure, it is impossible to foresee all ways people might want to use their computers in. But that's not a problem. We cover what makes sense for the base system. If people run complex application servers such as Java on top of them they can do that, there's no need in centralizing their process management too. We do not want to lose focus.

systemd is a very dynamic in itself, however it is not a vehicle to make other daemons more dynamic. It is also not a humongous abstraction effort to encode the entire system state in a single place. Instead, we focus on mount, device, process management and a few things around it. But that's it. We don't do lose focus and we want to have a very clearly defined set of goals what we want to do and what not. Or to put this in different words: if applications want to follow power states or track location then they should not talk to systemd about this, they should go directly to upower, to geoclue or whatever other service might be a good candidate.

How did this guy test systemd exactly?

Posted Oct 11, 2010 22:52 UTC (Mon) by nix (subscriber, #2304) [Link]

We don't do lose focus and we want to have a very clearly defined set of goals what we want to do and what not.
The phrase you're looking for I think is 'do one thing and do it well', which is of course very much the Unix way.

How did this guy test systemd exactly?

Posted Oct 12, 2010 7:16 UTC (Tue) by paulj (subscriber, #341) [Link] (10 responses)

By socket proxy I meant as proxy until the child could be spawned, of course - as the mentioned inetd and rmid do (well, iirc, rmid can also fully proxy).

It's good to hear there is a clear focus for systemd, and that it's not intended to displace higher layers. That, along with the DBus integration, makes me feel a lot better.

Still though, I don't understand the part of your post where you insist process creation belongs in PID 1, given you acknowledge lots of stuff wont be ported. :) Spawned processes can (and must, to properly daemonise, as you've written) reparent themselves to PID 1. Why can't systemd provide much of its baby sitting services (crashes) for those too? Also, clean environment, and the other steps detailed in the systemd daemon man page surely can be definitely be done in a library and would have the widest reach, so why not? Further, cgroups could also be setup by other processes (I'm slightly unclear on whether cgroups can be created or spawned into by unprivileged processes though).

Again, I am glad to hear systemd will stay focussed. I think it's important not to fall into the trap of trying to support everything, and fall victim to an ever growing amount of options, dizzying and intractable for the admin to comprehend (and no doubt for the maintainer too, to an extent). I think systemd maybe already has a toe in that hole, given all the funny socket-option options already there.

It might be an idea to push socket creation back to the services, so that systemd doesn't need all these options. E.g. have systemd run a binary that does whatever and hands back the fd to it over a pipe. This binary can be supplied by / packaged with the application. This would allow systemd to support more services without it itself having to be updated, where services need funny kinds of sockets or options that systemd doesnt do, and so make systemd more flexible still.

Anyway, thanks for taking the time to respond.

How did this guy test systemd exactly?

Posted Oct 12, 2010 14:50 UTC (Tue) by mjthayer (guest, #39183) [Link] (8 responses)

> It might be an idea to push socket creation back to the services, so that systemd doesn't need all these options. E.g. have systemd run a binary that does whatever and hands back the fd to it over a pipe. This binary can be supplied by / packaged with the application. This would allow systemd to support more services without it itself having to be updated, where services need funny kinds of sockets or options that systemd doesnt do, and so make systemd more flexible still.
Wasn't the whole point to load as few processes as possible in the early boot stages? If you load one babysitter process per daemon surely you are back to where you started - and for that matter, since many of the daemons do need to do exactly the same thing, having one application to handle that for all of them seems like a good thing to me. I suppose that systemd could have a mechanism for adding helpers for daemons that needed to do things slightly differently and that could really not be adjusted to work with it, although even there it would be nicer if such a helper could be shared by many daemons instead of each having their own one.

How did this guy test systemd exactly?

Posted Oct 12, 2010 15:05 UTC (Tue) by paulj (subscriber, #341) [Link] (7 responses)

Well, the common-case stuff can stay in systemd, and be fast. The odd-ball stuff can be handled by the binary. That avoids having to update systemd with options for every possible socket and socket option, before apps can use them. The alternative is that the application *cant* use systemd at all, which seems worse surely.

E.g. I maintain some daemons that could make use of demand launching, but systemd doesn't support the sockets and/or the options I need.. Hence why I suggested it.

How did this guy test systemd exactly?

Posted Oct 12, 2010 15:17 UTC (Tue) by mjthayer (guest, #39183) [Link] (1 responses)

> E.g. I maintain some daemons that could make use of demand launching, but systemd doesn't support the sockets and/or the options I need.. Hence why I suggested it.
Might that not have been a good place to start the discussion? I'm sure that if you got in touch with Lennart and discussed those specific issues then the two of you would come up with a good solution that might well be useful for other people too.

How did this guy test systemd exactly?

Posted Oct 12, 2010 16:51 UTC (Tue) by paulj (subscriber, #341) [Link]

The suggestion was made in a reply to Lennart, fwiw. ;)

How did this guy test systemd exactly?

Posted Oct 12, 2010 18:35 UTC (Tue) by mezcalero (subscriber, #45103) [Link] (4 responses)

Well, thankfully new sockopts are only added every now and then to the kernel. I don't think we'll have a hard time keeping up with that.

Can you elaborate on which socket types and options you are missing in systemd right now?

How did this guy test systemd exactly?

Posted Oct 13, 2010 13:04 UTC (Wed) by paulj (subscriber, #341) [Link]

Raw sockets. + there are various socket options I need to have set on connectionless sockets (e.g. multicast memberships; various meta-data retrieval options which might need to be set before packet is put into the socket). Routing stuff tends to use a variety of far-out options (and new ones come along not too infrequently - e.g. GTSM). ;)

Further, if I can specify these things in code, rather than in a config file, then I can share that code between the "setup socket(s) for systemd" binary and the main body of code. Which would help reduce redundancy and hence mistakes.

I'd like to say "I'll send you a patch", but my TODO list is several kms long at this stage. :(

How did this guy test systemd exactly?

Posted Oct 13, 2010 13:12 UTC (Wed) by paulj (subscriber, #341) [Link] (2 responses)

Oh, some specific, potential candidates:

- raw sockets
- meta-stuff like IP_PKTINFO, SO_BINDTODEVICE
- multicast stuff, IP_MULTICAST_LOOP, IP_ADD_MEMBERSHIP, IP_MULTICAST_TTL

To be clear, I think it'd probably be ungood for systemd to try explicitly support every possible socket type and option that services might want to use. ;) I'd prefer a coded offload mechanism.

How did this guy test systemd exactly?

Posted Oct 13, 2010 15:06 UTC (Wed) by mezcalero (subscriber, #45103) [Link] (1 responses)

SO_BINDTODEVICE we actually support (and actually in a pretty nice way, so that implicit deps on the network device are created, so that this actually works race-freely).

By "raw" sockets you mean raw IP sockets? Raw IP sockets usually also mean installation of a BSD socket filter, and I am not sure I want to include something like that in systemd.

AFAIK IP_PKTINFO is something that can be set when the app first takes possession of the socket. There's no need to set it before listening.

Apple's launchd has multicast support. I'd be willing to add that to systemd too, but I wanted to wait for a real-life usecase. Avahi (which I maintain) is probably the heaviest user of mcast on default Linux installations right now, and I don't think for Avahi it would make sense to have this.

How did this guy test systemd exactly?

Posted Oct 13, 2010 16:02 UTC (Wed) by paulj (subscriber, #341) [Link]

You can bind raw sockets to a specific IP protocol, and/or multicast groups. So you don't need a BPF filter per se. I'd have to read the kernel code for PKTINFO, but it wouldn't surprise me if the kernel marshalled the ancillary meta-data for the packet at the point of copying it into the socket buffer - and that it might not be available if you set PKTINFO later. I don't know though, havn't checked. ;)

Anyway, the point is, we shouldn't need this conversation - it'd be nice if apps could take care of the weirder stuff themselves by supplying their own binary to do the setup. ;)

Use cases: certain link-info and routing protocols. E.g. imagine you plug into a network that sends default route info via some routing protocol. Rare case, but imagine if systemd could auto-launch the right daemon.. Link-info protocols can clues about location and the like.

How did this guy test systemd exactly?

Posted Oct 12, 2010 18:32 UTC (Tue) by mezcalero (subscriber, #45103) [Link]

If other packages want their software supervised by systemd they can just drop static service files into /lib/systemd/system and/or dynamic service files into /etc/systemd/system. And then they can tell systemd to start/stop/whatever the service as they wish via simple D-Bus calls.

I'd rather wait for those "oddball" services to show up and become a problem before I'd think of shifting socket creation into any external binaries.

So far we didn't have any problems with the current listening code, and Apple built an entire OS on this idea, so I think unless somebody shows me a real-life problem with this i rather not spend too much time thinking about non-existing problems.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds