LWN.net Logo

Scheidler: Project Lumberjack to improve Linux logging

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 1, 2012 17:19 UTC (Thu) by xxiao (subscriber, #9631)
Parent article: Scheidler: Project Lumberjack to improve Linux logging


(Log in to post comments)

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 1, 2012 18:06 UTC (Thu) by HelloWorld (guest, #56129) [Link]

Every time I see this idiotic petition I feel the urge to create another one asking for people to stop whining...

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 1, 2012 19:02 UTC (Thu) by mads (subscriber, #55377) [Link]

Agreed.

I cannot fathom why anyone could wrap their minds around the concept of _making a petition_ to stop someone ... contributing.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 16:14 UTC (Fri) by marcH (subscriber, #57642) [Link]

Contributing is not the problem - deprecating is.

There is no doubt that Lennart's work is disruptive. There is not much more doubt that Lennart's software does NOT follow Unix principles (as presented here http://catb.org/~esr/writings/taoup/). I'm pretty sure he made clear he considers the Unix philosophy to be a massive failure for the Desktop. Market share tends to support this assertion.

Whether it is possible to make a Linux system good enough for both the desktop and server spaces is an interesting question. Just like the people signing this (actually funny) petition, I'm sceptical.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 18:23 UTC (Fri) by drag (subscriber, #31333) [Link]

> There is not much more doubt that Lennart's software does NOT follow Unix principles

Really? A couple of examples would be helpful.

we can take a look at PulseAudio, for example...

> Rule of Simplicity: Design for simplicity; add complexity only where you must.

Well Linux audio is a cluster f of a mess to begin with. PA didn't create the mess, but it's gone a long way to clean it up for people that take advantage of it.

> Rule of Modularity: Write simple parts connected by clean interfaces.

As time has progressed it has become rather obvious that the OSS style of using Posix-style file syscalls is not adequate to meet the needs of many types of audio applications. The Alsa attempt at creating a massive API that provides all sorts of functions for both high level and low level applications is not appropriate either. Then the attempts at papering over the problem with a large number of different libraries and new APIs and new abstraction layers does nothing to really address the fundamental problems of Alsa or OSS.

So creating a program that handles the needs of the management of audio I/O and creating much simpler and much more robust audio interfaces for application designers to use.

Much of the complexity that remains can be easily contributed to people's desire to live life without PA and providing compatibility with applications written on older APIs.

It also does not try to compensate for bugs in other layers. If you have buggy drivers it doesn't try to work around them. It expects applications to do their jobs, and the kernel to do it's job, etc.

> Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.

Esd was too simple. Artsd tried to do too much. Jack is too specialized for a specific task. All 3 did nothing to address the problem of supporting 'legacy' APIs. The OSSv4 idea of folding the complexity required to solve the audio problems into the kernel is just making a already massive and complex and difficult to debug program (your kernel) a even bigger thing.

With Pulseaudio it has separated what was attempted to be solved, inadequately, but a massive array of libraries and abstraction layers into a single program seems to match this rule.

> Rule of Transparency: Design for visibility to make inspection and debugging easier.

Linux audio is a difficult nut to crack, but PulseAudio's debugging and troubleshooting information that is written to the terminal when it's attached is actually pretty useful.

> Rule of Robustness: Robustness is the child of transparency and simplicity.

PA may crash, which is a problem for any application. But the API is written in a manner that applications that use it have no problem re-attaching to the daemon when it restarts.

> Rule of Repair: Repair what you can — but when you must fail, fail noisily and as soon as possible.

If your running it in a terminal it'll tell you when it fails and why it fails.

I could go on to the other ones, but this is already a excessively long post.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 18:35 UTC (Fri) by marcH (subscriber, #57642) [Link]

> I could go on to the other ones, but...

Too bad because you clearly picked up the least relevant example: who cares about audio or anything "multimedia" in the playing fields where Unix has been traditionally successful?

> ... this is already a excessively long post.

Apply the Rule of modularity?

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 20:29 UTC (Fri) by drag (subscriber, #31333) [Link]

> Too bad because you clearly picked up the least relevant example: who cares about audio or anything "multimedia" in the playing fields where Unix has been traditionally successful?

I think that I picked a good example because it's something that Unix has been poor at. The reason it was not successful was not because it followed the 'unix philosophy'.

> Apply the Rule of modularity?

I guess you have zero to offer to the discussion at this point.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 3, 2012 20:29 UTC (Sat) by marcH (subscriber, #57642) [Link]

> I think that I picked a good example because it's something that Unix has been poor at. The reason it was not successful was not because it followed the 'unix philosophy'.

Indeed. The reason it was not successful is because the Unix philosophy is not a silver bullet.

> > Apply the Rule of modularity?

> I guess you have zero to offer to the discussion at this point.

Exactly why I was inviting you to continue your monologue. You seem keener on writing than reading. The more you write you more likely you will write something vaguely related to my initial post. And the unrelated is not uninteresting either.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 4, 2012 18:36 UTC (Sun) by drag (subscriber, #31333) [Link]

>The more you write you more likely you will write something vaguely related to my initial post

Oh Really?

> There is not much more doubt that Lennart's software does NOT follow Unix principles (as presented here http://catb.org/~esr/writings/taoup/).

Is this somebody else channeling through you then?

Because I swear I was disagreeing with you and pointing out how Lennart's design follows the 'Unix principles as outlined in taoup' better then most things.

Although the clever use of double negatives makes it tricky to follow your logic.

Maybe you meant that you think it does design his programs in a 'Unixy' manner?

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 19:42 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

Well, I've recently had to install tomcat on Android phone (don't ask). I was surprised, but I managed to pull it off.

Right now I actually think that Android would be able to support a server infrastructure much sooner than common Linux desktops could evolve into something usable for general users.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 3, 2012 12:37 UTC (Sat) by alankila (subscriber, #47141) [Link]

Cool hack. But hey, tomcat is supposed to be pure java, so I suppose dvm could work with it without changes. Did you have to do anything special to make that happen?

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 3, 2012 23:02 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

I had to cut quite a lot of stuff, Tomcat may be pure Java but there's a lot of stuff there which doesn't work in Dalvik.

Jetty is much nicer than Tomcat and has Android version: http://code.google.com/p/i-jetty/

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 20:41 UTC (Fri) by Wol (guest, #4433) [Link]

There is not much more doubt that Lennart's software does NOT follow Unix principles

What on earth is systemd, if not "do one thing and do it well"?

The whole point of systemd is to take all the scripts, that spend thousands of lines doing the same thing repeatedly (and doing it pretty badly), and replacing them with small C modules.

aiui, if you count loc in /etc/init.d, for any half-way sizeable system you're likely to be well into 5 or 6 significant figures. And that's lines of *code* not *comments*. Whether he's pulled it off or not, Poettering's aim was (*including* the source of all the helper apps) to shrink that down to a far smaller figure, possibly the low 4 figures.

And as somebody pointed out, one of the BIG problems with Poettering is he doesn't work around other peoples' buggy software. Which is why PA was such a mess to begin with - Poettering assumed the other programs he was relying on worked as advertised, and they didn't :-(

If you want to go on about bloat and avoiding "the tao of Unix", you really should be having a go at Gnome and KDE! On second thoughts, don't upset the KDE people, that's my desktop ...

Cheers,
Wol

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 5, 2012 9:25 UTC (Mon) by alankila (subscriber, #47141) [Link]

I think there should be the equivalent of Godwin's law about the "do one thing and do it well". I simply haven't yet seen a situation where it doesn't apply, because the "one thing" can apparently be chosen so flexibly that this phrase always works.

No matter how hideously complicated the program is, or no matter how obscure its purpose, if you just define that as the "one thing" the program is supposed to do, and are satisfied with how the program works, you can blurt out this sentence.

The original citation seemed to talk about programs that work together, and use text streams as the interoperability protocol. I get the impression that we should look at the citation as useful guideline for designing the primitives to be used in shell programming, which is actually something we don't really do that much anymore. The tools have evolved, this saying has not.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 5, 2012 12:16 UTC (Mon) by marcH (subscriber, #57642) [Link]

> I think there should be the equivalent of Godwin's law about the "do one thing and do it well". I simply haven't yet seen a situation where it doesn't apply, because the "one thing" can apparently be chosen so flexibly that this phrase always works.

And by the way: http://archive.fosdem.org/2011/interview/lennart-poettering
"Systemd touches a lot of things and since it manages the entire runtime behaviour of the Linux userspace there's enough stuff to talk about to fill many hours. At FOSDEM we can only cover a subset of it."
(Thanks for the link)

I am not at all interested in discussing how "Unixy" is Lennart's work. I already know it's way off, thanks. On the other hand, interesting are the non-passionate discussions about the limits of the Unix philosophy and about what you win and lose when you depart from the AOUP rules.

No matter how good or bad the omelet eventually tastes, you have to break eggs to make one. Lennart is probably a very good cook. For sure he is good at breaking eggs. Everyone please show some pity and understanding for angry birds.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 9, 2012 0:32 UTC (Fri) by sgros (subscriber, #36440) [Link]

I don't get you! Do you think that Unix will be the same in 50 years from now? Do you think that today's Unix is the same as 30 years ago? Do you think that the word "simple" has absolute meaning?

And even if the answer is yes to my questions, why are you bragging so much? No one forces you to use Fedora or any other distribution that embraced Poettering's work. In the end, you have BSDs so use them. The final verdict will be given by user base.

Oh, and Solaris has something similar to systemd, it's called smf. And Solaris used to be a role model for Linux for a long time (and still is).

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 5, 2012 12:56 UTC (Mon) by mpr22 (subscriber, #60784) [Link]

It's not do one thing, it's the important counterpart concept: do any given thing in one place.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 5, 2012 14:37 UTC (Mon) by alankila (subscriber, #47141) [Link]

I think gnu tools (for instance) violate that principle everywhere. Ls can sort, sort can uniq, find can xargs... I think it is fiction, one way or other.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 5, 2012 15:02 UTC (Mon) by dgm (subscriber, #49227) [Link]

And BusyBox can all of the above. Come on...

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 6, 2012 12:58 UTC (Tue) by jwakely (subscriber, #60262) [Link]

Those features are all required by POSIX, they're not specific to the GNU tools

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 3, 2012 11:28 UTC (Sat) by drago01 (subscriber, #50715) [Link]

> Contributing is not the problem - deprecating is. No it isn't. That's how you make progress. You deprecate old stuff and replace it with newer saner stuff. Lets take systemd as an example. It replaces and old system this is nothing more then a pile of hacks stuffed on top of each other with something well designed. I.E no code duplication all over the place, not starting 100s of processes to do simple things like parsing strings, not using code as configuration files, proper separation and supervising of services ... And no this does not only help "the desktop" but pretty much every Linux based system (yes that includes servers). Yes you have to learn how to use the newer systems but that's life. You can't just freeze everything due to laziness.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 3, 2012 20:42 UTC (Sat) by marcH (subscriber, #57642) [Link]

> > Contributing is not the problem - deprecating is.

> No it isn't. That's how you make progress.

Just like every other single discussion involving Lennart's work, this one shows very clearly the clash between two worlds: developers focused on internal design quality, versus users who do not give a s... about all this and only care about regressions.

What is probably the most frustrating with systemd is the switch to a compiled language. While this is a technically superior solution in many ways it removes *in practice* transparency and "hackability" for sysadmins (most of them comfortable only with scripting).

People surprised by this petition have never honestly tried to put themselves in the shoes of the average end lusers.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 4, 2012 0:35 UTC (Sun) by anselm (subscriber, #2796) [Link]

What is probably the most frustrating with systemd is the switch to a compiled language. While this is a technically superior solution in many ways it removes *in practice* transparency and "hackability" for sysadmins (most of them comfortable only with scripting).

I was under the impression that while systemd comes with C tools for all of the boot-time customisations and does not require shell scripts to manage services, the system administrator is free to stick in shell scripts wherever they please if they need something custom that the C stuff doesn't do. Systemd also supports SysV init scripts (with some loss of functionality compared to managing services natively in systemd) out of the box.

It seems to me that all the »hackability« is actually there – it's just not required in many cases because the infrastructure that systemd already provides is adequate. As far as »transparency« is concerned, systemd is also a big step forward since one does not need to analyse the init script for every single service separately to find out exactly how that service is started; looking at the (much shorter and simpler) systemd service file will suffice.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 4, 2012 9:10 UTC (Sun) by khim (subscriber, #9252) [Link]

It seems to me that all the »hackability« is actually there – it's just not required in many cases because the infrastructure that systemd already provides is adequate.

The problem here, of course, is the fact that most services will not come with SysV init scripts which can be “slightly altered”. It's the same story as with “PHP” vs “___ language”. The ability to easily change things in production is both really nice (and this is what “sysadmins” clamor for) and completely horrifying at the same time.

I welcome systemd with both hands exactly because I have worked as sysadmin and I know “both sides” of the equation. It's just not worth it. If you really need to hack something - you can do that with systemd. But this will be highly visible and traceable (you'll get new large file in mostly-empty directory, not two lines added to large pile of scripts god knows where).

As far as »transparency« is concerned, systemd is also a big step forward since one does not need to analyse the init script for every single service separately to find out exactly how that service is started; looking at the (much shorter and simpler) systemd service file will suffice.

I think “transparency” here means something like “the ability to insert "set -x" in your script to see what really goes on here”. Yes, systemd is more opaque in his work, which is kind of unfortunate, but this [small] problem is more then compensated by it's advantages: I'm saying this, again, as former sysadmin - the fact that Linux had no way to reliably kill service before systemd was created is just crazy.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 8, 2012 17:16 UTC (Thu) by sorpigal (subscriber, #36106) [Link]

I have a sort of love-hate relationship with both systemd and SysV. I like a lot of things about systemd, but the anti shell script crusade is a big turn off. I like that SysV is "simple" in that while there's a lot you need to do "just so" to do it right, the process is simple to explain and wrap your head around. With systemd it's harder to make sense of what its model of operation is, especially because it *does* handle all sorts of different cases with its different options. Eventually you can pore through the man pages and get the picture for any given piece and eventually I may grow familiar with all the options to the point where I find it easy to know what's happening, but meanwhile it sure doesn't *seem* as nice as "simply" reading through a shell script and everything it sources. And yeah, set -x makes tracking down problems a lot less mysterious than other init systems.

And then... sometimes I want to review "all messages emitted during boot" and find that I can't, then I curse sysvinit's idiocy, demand to know why I have to jump through hoops to answer questions like "Why was this process started and do I need it?" and at these times I smile fondly at systemd. Until something breaks and I open a config file which has a parameter that does who-knows-what set to a value that means who-knows-what will happen at who-knows when.

I never want to be sitting at a Linux box experiencing what I like to call "That Windowsy feeling." You know, the one you get when you realize "I can't tell what happened. It just happens, I'll never know why it happens, I'll never be able to understand what it's doing, and I can't think of a thing to do to diagnose it." As long as systemd never leaves me feeling like that then it's A-OK.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 4, 2012 19:41 UTC (Sun) by xxiao (subscriber, #9631) [Link]

http://monolight.cc/2011/05/the-systemd-fallacy/

"(…) So, get yourself a copy of The Linux Programming Interface, ignore everything it says about POSIX compatibility and hack away your amazing Linux software. It’s quite relieving!

– Lennart Poettering @ fosdem.org"

I hope he is not serious!

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 4, 2012 23:28 UTC (Sun) by anselm (subscriber, #2796) [Link]

The article that comment links to isn't the actual source of the quote; it's really an anti-systemd rant that, like so many we've seen, appears to be big on emotion but largely uninformed by fact.

It's probably better to go directly to the real source of this quote for context. It's from a FOSDEM interview at http://archive.fosdem.org/2011/interview/lennart-poettering.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 5, 2012 17:36 UTC (Mon) by dmarti (subscriber, #11625) [Link]

In the past few years I haven't seen anything developed for POSIX -- un*x-ish code is either for Linux only, or more commonly Linux officially, plus some developer fixes anything that won't build on Mac OS so that they can test on their laptop. Is anyone targeting legacy Unix any more?

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 10, 2012 20:58 UTC (Sat) by lacos (subscriber, #70616) [Link]

I wrote lbzip2 (the 0.x series) for SUSv2 exclusively ("UNIX 98").

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 1, 2012 19:54 UTC (Thu) by ovitters (subscriber, #27950) [Link]

Nice list of people who think stop energy is something good. Rather have constructive people. Stop energy its dismissed quite easily and ineffective.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 1, 2012 19:57 UTC (Thu) by josh (subscriber, #17465) [Link]

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 2:28 UTC (Fri) by bronson (subscriber, #4806) [Link]

Now THAT's a petition I can sign. Heck with SOPA/PIPA/Patents/etc. :)

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 1, 2012 19:05 UTC (Thu) by drago01 (subscriber, #50715) [Link]

That's one of the most stupid petitions I have ever saw.

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 1, 2012 19:19 UTC (Thu) by intgr (subscriber, #39733) [Link]

I find it ironic that the website's name is "change.org", but the petition is trying to *resist* change. :)

Scheidler: Project Lumberjack to improve Linux logging

Posted Mar 2, 2012 19:11 UTC (Fri) by JEFFREY (subscriber, #79095) [Link]

+1

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds