User: Password:
|
|
Subscribe / Log in / New account

you do NOT need to write all your programs together to make them work together.

you do NOT need to write all your programs together to make them work together.

Posted Jan 28, 2013 23:23 UTC (Mon) by dlang (subscriber, #313)
In reply to: you do NOT need to write all your programs together to make them work together. by davidstrauss
Parent article: Poettering: The Biggest Myths

> For example, we added the journal with a new API and a compact on-disk format. Necessarily, we have to have a new daemon to receive messages (journald), a new tool for browsing the files as text streams (journalctl), and a new C header for doing the structured logging. The journal would be rather useless for a long time if added in program by program over years. Nothing in this design is more monolithic or less Unix-y than what it replaced, but it did require replacing all of those parts at once to get end-user value.

and why did you need to change all of those things?

syslog daemons already supported outputting to many different on-disk formats, why should you be requiring a different format?

Why invent a new way of doing structured logging? there were already N standard ways of doing structured logging, why did you have to invent a new one for systemd?

etc....


(Log in to post comments)

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 0:00 UTC (Tue) by davidstrauss (subscriber, #85867) [Link]

> syslog daemons already supported outputting to many different on-disk formats, why should you be requiring a different format?

None of those approaches supports field-structured logging because syslog has no method for handling such messages being sent to the daemon, let alone preserving the data on disk after receipt. You could have syslog write to an RDBMS like SQLite for all it matters, but if the structure's already gone, the structure's already gone.

The journal format also has a number of nice attributes like FSS, compression, atomic rotation, and indexing. And it's all available as friendly text streams in many formats using journalctl.

> Why invent a new way of doing structured logging? there were already N standard ways of doing structured logging, why did you have to invent a new one for systemd?

What are those N ways? Certainly, syslog doesn't support field-structured logging. Protocols like GELF are asynchronous and lossy by design. Java's logging tools have obvious lack of usability on many embedded systems. Tools like Flume are designed to harvest logs off disk after they've already been written.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 1:16 UTC (Tue) by HelloWorld (guest, #56129) [Link]

dlang has been spreading falsehoods and refused to educate himself about the design rationale of systemd, the journal etc. so many times that I suggest you stop wasting your time.
Haters gonna hate, that's life.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 1:30 UTC (Tue) by mgb (guest, #3226) [Link]

Field structured logging might be useful for some people.

So write a field-structured logger and allow people to use it. That's the UNIX way. That's the way that allows UNIX to progress.

Don't leverage your control of pid 1 to force people to use your logger. That's the Poettering way. That's the way that stifles UNIX progress.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 2:06 UTC (Tue) by raven667 (subscriber, #5198) [Link]

> Field structured logging might be useful for some people.

True

> So write a field-structured logger and allow people to use it. That's the UNIX way. That's the way that allows UNIX to progress.

True, and that well describes the systemd approach

> Don't leverage your control of pid 1 to force people to use your logger. That's the Poettering way. That's the way that stifles UNIX progress.

False! I don't know how you got here, no one is forced to use the Journal, it does get a copy of the local logs but they go to the normal syslog daemon too and are written out normally or sent across the network normally.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 2:34 UTC (Tue) by mgb (guest, #3226) [Link]

See e.g. http://rpmfind.net//linux/RPM/fedora/18/i386/s/systemd-19...

systemd REQUIRES libsystemd-journal.so.0

You are Lennart Poettering and I claim my £5.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 3:59 UTC (Tue) by raven667 (subscriber, #5198) [Link]

I am not Lennart Poettering but I'm flattered you think so 8-) No £5 for you. Pro tip: Poettering is mezcalero on LWN.

In any event I didn't dispute that systemd logs to the Journal, just that _YOU_ aren't required to use it because everything still goes to traditional syslog. There is no credible way you could claim that anything is being stifled when compatibility is such an obvious priority.

I'm not sure we are really communicating effectively because you don't seem to be understanding what I am saying and your responses don't make any sense to me. I will assume that you are operating in good faith, bid you a good day and thank you for your time, I don't have anything else to say.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 11:17 UTC (Tue) by nix (subscriber, #2304) [Link]

To be honest, if the default journal is an in-memory passthrough, I don't see why anyone would care, anymore than anyone says that syslog-ng has an 'extra useless log' because it does in-memory buffering to keep disk I/O down.

If one said that systemd had an *optional* field-structured message log, would this suddenly make this problem go away, even though the only difference is to change terminology and declare the journal 'off' rather than 'in-memory'?

(The only thing I really disliked about the journal was the explicit lack of definition of the binary journal format. Since this has now been rectified, my complaints about it are gone.)

you do NOT need to write all your programs together to make them work together.

Posted Jan 31, 2013 19:07 UTC (Thu) by sorpigal (subscriber, #36106) [Link]

> If one said that systemd had an *optional* field-structured message log, would this suddenly make this problem go away, even though the only difference is to change terminology and declare the journal 'off' rather than 'in-memory'?

I suspect a great deal of the push-back against systemd comes from this very thing. Given the option of talking about it in a way that doesn't rile people up and talking about it in a way that does, the latter is what happens. If the story were "systemd internally uses its own field-structured log format but still outputs logs to syslog by default (or you can just use the internal format directly)" then you'd see more acceptance, but the story was "systemd has replaced syslog with a new, undocumented, non-text log format." If the goal wasn't to annoy people then I can only say that there exists an incredible natural talent for annoyance which is being wasted in the field of software engineering.

you do NOT need to write all your programs together to make them work together.

Posted Jan 31, 2013 19:41 UTC (Thu) by dlang (subscriber, #313) [Link]

remember, this isn't just systemd's internal logs, systemd intercepts logs that the apps write to syslog and turn them into this new structure as well.

you do NOT need to write all your programs together to make them work together.

Posted Jan 31, 2013 20:42 UTC (Thu) by davidstrauss (subscriber, #85867) [Link]

> remember, this isn't just systemd's internal logs, systemd intercepts logs that the apps write to syslog and turn them into this new structure as well.

Just keep omitting that systemd's journal forwards the entries *unaltered as syslog*, and you might convince some people that the journal breaks their traditional syslog toolchain.

you do NOT need to write all your programs together to make them work together.

Posted Jan 31, 2013 20:46 UTC (Thu) by dlang (subscriber, #313) [Link]

look at the context, I am replying to someone explaining how presenting how the journal deals with logs differently could avoid antagonizing people. I am just pointing out that his statement is not complete, it's not just the internal journald logs that are involved.

you do NOT need to write all your programs together to make them work together.

Posted Feb 1, 2013 1:21 UTC (Fri) by davidstrauss (subscriber, #85867) [Link]

> look at the context, I am replying to someone explaining how presenting how the journal deals with logs differently could avoid antagonizing people. I am just pointing out that his statement is not complete, it's not just the internal journald logs that are involved.

I'm aware of the context. I just think it's disingenuous to use how the systemd's journal maintains its own data structures to imply some change happens that creates complexity for syslog users.

Use of internal data structures is true of any log daemon, including rsyslog. Saying that the journal "turn[s] them into this new structure as well" implies that the syslog messages emitted by the journal are not identical to the messages sent in, which is false.

The journal goes beyond syslog, but it does not get in its way or force users to change any existing applications, monitoring, or workflows.

When we updated our configurations for Fedora 17 (the first release with the journal), we didn't have to touch anything to continue getting the same syslog messages we got before in the same places we had always looked for them.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 7:59 UTC (Tue) by smurf (subscriber, #17840) [Link]

Oh come on.

So you want to force the systemd people to support writing two log streams? One for traditional syslog and one for field-structured logging? In what way does that fit your UNIXy "do one job well" definition?

Instead, systemd spits out a nice structured log, journald reads that and translates it to "standard" syslog, other programs can be switched over to structured logging once journald can be assumed to be available (or they can support both if they want the compatibility, or somebody can write a compatibility library with a journal API and syslog output), everybody is happy.

Why the hell should it matter that systemd and journald are maintained in a single repository?

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 17:35 UTC (Tue) by raven667 (subscriber, #5198) [Link]

I'm guessing that most software never will switch to using the Journal and will continue using standard syslog but that's fine and is supported.

you do NOT need to write all your programs together to make them work together.

Posted Jan 29, 2013 20:43 UTC (Tue) by khim (subscriber, #9252) [Link]

Yes, it is supported but in sane manner: systemd only creates log in journald format and journald converts them. Somehow people who like to raise racket about "monolithic design" and "lack of modularity" simultaneously find this solution problematic and want to stuff syslogd format support in systemd itself. Talk about consistency.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds