| |||||||||||||
Poettering: systemd for Administrators, Part VIIIPoettering: systemd for Administrators, Part VIIIPosted Apr 21, 2011 19:01 UTC (Thu) by wahern (subscriber, #37304)In reply to: Poettering: systemd for Administrators, Part VIII by dskoll Parent article: Poettering: systemd for Administrators, Part VIII
Like this?
Though, that uses the shell, which apparently is anathema these days.
(Log in to post comments)
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 19:40 UTC (Thu) by alvieboy (subscriber, #51617) [Link]
Avoiding unnecessary (avoidable) forks is always a good thing.
opendir(),readdir() and friends can accomplish this easily. Or just provide a "cached" version which includes all of your files.
Or better, have something like dnotify() updating that cache for you.
People often forget that is not the shell itself that is slow - but rather the amount of commands (with all overhead from disk IO, dynamic linking and so on) that goes with it.
Ah, and unnecessary caching which might cause swapouts. I never actually managed to disable read cache, even with {m,f}advise().
Al
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 20:30 UTC (Thu) by wahern (subscriber, #37304) [Link]
We're talking about reading configuration files, not writing a web server.
One of my fantasy projects is to write a new shell. The core design idea is to forgo forking. Not because forking is slow per se, but so there can be other optimizations. Instead of forking commands, any time a unique PID is required a slave process from a cached pool will be used to direct signals to the executing process. (Or, perhaps, a kernel extension written for the creation of light-weight PID-only processes.)
With most commands running as coroutines in the same process, you can start adding new features and commands to the shell language not feasible when running a collection of separate processes. Similarly, routines like system() and popen() could be run inside the same process.
The tentative name will be "Project Everything Shell".
It'll compile programs for a simple virtual machine, eventually sporting JIT compilation. Like Java the opcodes will be tailored for shell features, with opcodes for psuedo-forking, etc. In a sense it'll be a tiny Unix-oriented virtual machine.
It won't include interactive features, but because it will be written as a library it will be easy to build an interactive shell. That's a whole 'nother fantasy, though.
I'm not even joking. I'm going to make this happen. Eventually....
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 22:11 UTC (Thu) by HelloWorld (guest, #56129) [Link]
So what does your "new shell" do that several high-level programming languages don't already do today?
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 0:40 UTC (Fri) by wahern (subscriber, #37304) [Link]
Adhere to POSIX shell syntax and semantics so that it can be a drop-in replacement for existing shell scripts.
The shell mirrors the Unix process model very closely, and few other languages do that. That's what makes it useful. The closest example I can think of is Tcl, but it's no more POSIX-shell compatible than any other language.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 3:21 UTC (Fri) by HelloWorld (guest, #56129) [Link]
> Adhere to POSIX shell syntax and semantics so that it can be a drop-in replacement for existing shell scripts.
In that case, I'm not even remotely interested. The traditional Unix Shell model is utterly broken by design, and it's impossible to fix it without incompatible changes.
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 22:33 UTC (Thu) by cmccabe (guest, #60281) [Link]
> One of my fantasy projects is to write a new shell. The core design idea
> is to forgo forking. Not because forking is slow per se, but so there can > be other optimizations.
What other optimizations were you thinking of?
You can already share memory between processes. You can already pass messages between processes without too much overhead. You can already control one process with another process.
Cramming unrelated stuff into a single process makes the kernel scheduler's job harder (in some cases impossible to do well, like on multicore machines). It gets rid of the safety net that a separate address space gives you. Now a bug in your application can mutate the shell's memory and look like a shell bug. Fun fun!
If anything, we need to be moving in the direction of more parallelization and more isolation between components. UNIX got it right the first time around-- and using hardware we wouldn't put in a toaster oven nowadays.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 0:32 UTC (Fri) by wahern (subscriber, #37304) [Link]
What other optimizations? Optimizations which mitigate the performance hit that people are complaining about. For example, exec costs like dynamic linking (static linking of /bin binaries never caught on). Also, you could add speculative loading of shell script commands. I'd want to add useability features like exposing file locking and PID management primitives in a way that makes them as convenient to use as in C. Basically, the idea is to solve all the little issues that make people stop using the shell for trivial system management functions which *should* be using the shell.
The shell's raison d'etre was process and file management. Don't you find it odd that all of sudden people are successfully arguing that the shell sucks at process and file management?
I agree we should be moving in the direction of more process isolation. But projects like systemd are moving in the *opposite* direction. init(1) shouldn't be a complex process, and now at least on Linux it's becoming a behemonth. At some point systemd will sprout a scripting language and history will repeat itself. So if we're going to go crazy, I want to join the party.
My idea wouldn't run all system scripts on a single process. There's no mixing of user privileges so security isn't as much of an issue. Concerns regarding bugs apply equally or more so to systemd. And the behavior of shell commands naturally fit a cooperative concurrency model for which coroutines would be a good fit. Plus, the type of shell I'm talking about could fix the issues with control over long pipelines that current shell designs don't or can't easily fix, such as access to exit status.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 1:35 UTC (Fri) by cmccabe (guest, #60281) [Link]
> Optimizations which mitigate the performance hit that people are
> complaining about. For example, exec costs like dynamic linking (static > linking of /bin binaries never caught on).
I'm pretty sure stuff like preload(8) has this covered.
> I'd want to add useability features like exposing file locking and PID
Bash already exposes file locking. man flock(1).
PID file management is going to be integrated with systemd, rather than exposed in shell scripts like it is now.
> Plus, the type of shell I'm talking about could fix the issues with
Bash has $PIPESTATUS.
> I agree we should be moving in the direction of more process isolation.
No matter what you think about systemd, you can't deny that it runs daemons in a separate process. And that is a very Good Thing.
Personally, I think systemd will work out well in practice. But I haven't had a chance to try it out yet.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 2:11 UTC (Fri) by wahern (subscriber, #37304) [Link]
systemd will work well in the short run. But give me an example where moving a hundred different tasks into a single, monolithic C application with a descriptive or declarative configuration format has worked out in the long run.
Everything that systemd replaces is the result of endless experimentation made possible by the flexibility that scripting languages--particularly and especially shell scripting--allows. Systemd will now destroy this process. Systemd won't stop the inevitable hacks that people will implement in the future; it's only cleaned up the past hacks. Future hacks are going to be uglier, more obscure, and more frustrating because they're going to include dreadful workarounds to the rigidity of systemd. That is, unless and until systemd reinvents the wheel regarding scripting support.
Come back to this thread in 10 years, tell me I was wrong, and I'll buy you a beer.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 4:39 UTC (Fri) by josh (subscriber, #17465) [Link]
You can always patch systemd if you don't like its behavior. Or you can install a systemd file that invokes a shell script if you really want to.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 10:13 UTC (Fri) by mpr22 (subscriber, #60784) [Link] Tell me one thing you would want to do which (a) would need to be managed by one of the things systemd replaces and (b) would be impossible or ugly with systemd. Here's the man pages.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 12:45 UTC (Fri) by mezcalero (subscriber, #45103) [Link]
You are very confused.
The early boot stuff systemd covers is not done from a single process. It's all neatly separated in individual binaries. Nice, modular, simple and parallelizable. All those binaries are simple C programs with no deps but libc. So it's not monolithic at all.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 17:25 UTC (Fri) by martinfick (subscriber, #4455) [Link]
Perhaps you should think of systemd as a "little language"? I do.
http://c2.com/cgi/wiki?LittleLanguage
And, from that perspective, systemd is something well overdo. Unix has many little languages to make simple things simple, and my impression of systemd is that it makes many more simple things simpler.
I love the power of shells, but I cannot endorse what standard linux startup scripts have become. They have always seemed to me to be a place ripe for the introduction of a little language, and I am shocked that it has taken so long! Now that someone has finally done so, and improved on system isolation and dependency management, without taking away the power of using a shell, I am delighted.
I feel that systemd embodies the unix philosophy way better than some tangled mess of shell scripts do (and way better than other startup systems on other OSes). I hope that Lennart continues to push solutions to fill the linux gaps that he sees, particularly the ones which the oldbeards can no longer see because they are so used to overlooking them. Thank you Lennart!
Poettering: systemd for Administrators, Part VIII Posted Apr 28, 2011 15:56 UTC (Thu) by stevem (subscriber, #1512) [Link]
Absolutely: loss of flexibility, loss of experimentation, loss of tweakability. Ick.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 15:58 UTC (Fri) by vonbrand (subscriber, #4458) [Link] All the system calls, and a good load of "system" libary functions, are available (almost) directly in Perl (and I'd guess in other such languages)...
Poettering: systemd for Administrators, Part VIII Posted Apr 28, 2011 23:45 UTC (Thu) by steven.blake (subscriber, #13731) [Link]
We could probably use the original Unix hardware as a toaster oven.
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 22:42 UTC (Thu) by cmccabe (guest, #60281) [Link]
P.S. Sorry to be so negative. I think there are a lot of cool improvements that you could probably make to the shells that are out there; just not the one you described!
C.
Poettering: systemd for Administrators, Part VIII Posted Apr 27, 2011 12:34 UTC (Wed) by engla (guest, #47454) [Link]
Is it worth it over just improving busybox?
I'm not saying that busybox should include yet more utilities, but the ones it has provide a good basis for a non-forking shell.. for the important parts.
But if you could help, the POSIX compliance of busybox' shell needs some improvement to be perfect.
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 23:46 UTC (Thu) by dgm (subscriber, #49227) [Link]
> voiding unnecessary (avoidable) forks is always a good thing.
Is always a good thing to trade simplicity and robustness (a single line of code) for a very tiny speedup?. I don't think so.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 0:36 UTC (Fri) by wahern (subscriber, #37304) [Link]
You're confusing layers here. fork() is a single line of code in C, but hundreds or thousands of lines in the kernel.
Likewise here, the shell would be POSIX compatible. So you don't sacrifice simplicity at the usage level. My aim is only to address all the feature and performance problems that are driving people to monolithic applications like systemd.
And to do that, new code has to be written somewhere. I believe that there could be less overall complexity in the long run by sticking to shell scripting, rather than moving to all of these unique, complex applications.
Poettering: systemd for Administrators, Part VIII Posted Apr 22, 2011 4:13 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]
Why do we still need to experiment with the way daemons are started? It should be just solved once and for all.
Poettering: systemd for Administrators, Part VIII Posted Apr 23, 2011 22:27 UTC (Sat) by wahern (subscriber, #37304) [Link]
Well, for one thing, as Lennart discussed in his previous article in this series, the daemon manager cannot do all of the privilege separation and environment boxing for the application. So any sufficiently sophisticated daemon will have to do a substantial amount of the work it's already doing, or should be doing.
The work that systemd actually does for daemonizing is about ~30-50 lines of C code. Code which I already have to write because I need my applications to be portable.
Anyhow, I don't understand why people are arguing about changing the methodology of how systemd is daemonizing applications. My idea is merely to make the capabilities that systemd uses to manage services available from a shell-compatible environment, so all of that logic isn't hard coded in C. Inevitably the declarative configuration format systemd uses will rub some people the wrong way, and we'll be back to square zero in terms of dealing with ugly workarounds. Better to let people write workarounds so that they're less ugly, rather than forcing them into a rigid framework which presumes that we've reached the end of the road in terms of how services are managed, locking in current practice.
But this is all partly tongue in cheek. I have every intention of writing a shell like this, but I don't kid myself that it will ever find adoption.
Poettering: systemd for Administrators, Part VIII Posted Apr 24, 2011 2:00 UTC (Sun) by mpr22 (subscriber, #60784) [Link] When I look at systemd's man pages, I don't see all that much rigidity. Could you cite (with reasoning) some specific aspect of its rigidity that you think is going to cause problems, or are you just doomsaying?
Poettering: systemd for Administrators, Part VIII Posted Apr 24, 2011 2:44 UTC (Sun) by wahern (subscriber, #37304) [Link]
It's rigid by it's nature because it's not programmable. But off the top of my head I'll give you an example that browsing through the man pages I can't find a solution for:
Say you have an error in your configuration file. The daemon bails with an error message. Where does this error message go? I'm sure everyone in this thread has experienced some frustration when a service which supposedly started--because the service manager returned successfully from the start command--was nowhere to be found. If the error message went to the terminal that started the service you would immediately see the error. But if stderr is being redirected to a service manager, this isn't going to happen. And there's no easy solution to this; maybe the configuration error doesn't happen for a few seconds because some other initialization is occurring (e.g. long'ish DNS lookup or disk access), so how can the service manager reliably know when to redirect errors?
In my daemons all error messages go to stderr, but stderr isn't redirected to a log file--if it's redirected at all--until after initialization of all the sub-components. So, for example, sockets are bound *before* forking (yes, please no comments that systemd can bind sockets early too, this is just a familiar example) and all other initializations are ordered to make sure "fail early" happens and detaching from the terminal occurs as late as possible to minimize annoying surprises.
Does systemd address this scenario? In my fantasy world this would be addressed somehow, but I don't have the perfect answer to how. Sure there are solutions you can dream up, but will it be the right one? With systemd there can be no experimentation with messaging the service manager to see where the best solution lies. Whoever gets their patch upstream first wins the day. This is the sort of rigidity I'm speaking of. The very fact that you have a non-programmable framework is the barrier, not how well or comprehensive it's implemented.
If you've ever worked at a company that builds embedded appliances then you would agree that there is no one-size-fits-all approach to service management. But for any service to be portable across such Linux environments--disregarding other Unix systems--then systemd doesn't really solve much of anything for developers, because there will have to be the usual boilerplate code and/or shims in place, regardless.
Poettering: systemd for Administrators, Part VIII Posted Apr 24, 2011 23:43 UTC (Sun) by mezcalero (subscriber, #45103) [Link]
Services should log there error causes to syslog. They can do that either via the syslog() libc call or by simply writing to STDOUT/STDERR, and using StandardOuput=syslog in the service file.
In F16 we plan to set StandardOutput=syslog by default. We are currently holding this back in order to support generic syslogs with SysV init scripts where such a default would result in a logging loop.
Poettering: systemd for Administrators, Part VIII Posted Apr 29, 2011 4:08 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
and what do yo do before syslog is running?
or more to the point, where to the logs go if it's the syslog daemon that's having the problem?
Poettering: systemd for Administrators, Part VIII Posted Apr 25, 2011 9:44 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
Well, I _am_ working (actually, owning) a small company that does embedded development.
And I'm looking at systemd to replace our current SysV-based infrastructure. Systemd is faster, smaller and more robust.
We've tried upstart earlier but while it has some nice features, it just isn't worth the cost.
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 22:19 UTC (Thu) by dskoll (subscriber, #1630) [Link] Sure, under the hood it could be implemented with popen/pclose just as long as it's done securely and portably. The shell command line should be hidden because it's an implementation detail.
Poettering: systemd for Administrators, Part VIII Posted Apr 21, 2011 22:22 UTC (Thu) by dskoll (subscriber, #1630) [Link] Following up on myself... the reason I did not do it in one of my projects is that it's handy to be able to print the file name and line number in the event of an error. So my proposed API is far too simple, but it would be nice to have some standard API that makes it convenient to iterate over multiple config.d files yet still gives you access to file name and line number for diagnostics.
Poettering: systemd for Administrators, Part VIII Posted Apr 28, 2011 1:48 UTC (Thu) by elanthis (guest, #6227) [Link]
That's just far too application specific.
Once you get into dealing with line numbers you're really dealing with the innards of your parser.
And while I'd _love_ for there to be a standard UNIX configuration file format instead of the current nightmare where every damn app uses its own format (and often a different format for different individual config files used by the same app!), that just ain't going to happen.
What you can do though is look into something like libconfig or libini or one of the dozen similar libraries and get foo.d support added there. It may not be a "standard" but it's at least a reliably available library you can depend on and use.
Poettering: systemd for Administrators, Part VIII Posted Apr 28, 2011 15:59 UTC (Thu) by dskoll (subscriber, #1630) [Link] Once you get into dealing with line numbers you're really dealing with the innards of your parser. Not at all. You really just need a way to know which file you're currently reading from. (You can keep track of line numbers on your own.) An API that didn't expose the file name would be horrible for producing diagnostics.
|
|||||||||||||