From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
Posted Feb 1, 2014 8:30 UTC (Sat) by fandingo (guest, #67019)In reply to: From anti-systemd to pro-systemd in the shortest time by dlang
Parent article: This week in "As the Technical Committee Turns"
If a journald network sender were implemented, your complaints about scale would almost certainly disappear completely. Obviously, there would be no memory copies to send a message from journald to an external sender (like rsyslog). Additionally, it's already trivial to avoid *any* storage by specifying Storage=none.
Posted Feb 1, 2014 8:57 UTC (Sat)
by dlang (guest, #313)
[Link] (10 responses)
why is it that journald refuses to pass the metadata on to a syslog daemon? that would be a lot simpler.
and even if the journal could send the logs over the network, there is still a lot that syslog daemons can do in terms of filtering that journald can't.
why is it NIH if the syslog daemons try to work around journald to keep doing their job, but it's not NIH for journald to throw away decades of experience and code to do things from scratch?
Posted Feb 1, 2014 9:15 UTC (Sat)
by mchapman (subscriber, #66589)
[Link] (9 responses)
Isn't that precisely what Rsyslog's imjournal module is for?
If you're question is "why can't journald pass on the metadata as part of the syslog message themselves?" then you're going to have a whole lot more complexity in deciding how such metadata should be formatted, and that's probably a lot of config that has no rightful place in journald itself.
Remember, the journal can contain pretty much *arbitrary* metadata, whatever applications might decide to store there, so there's really no standard way of formatting it all. This is the 3/4 fandingo was talking about. As an example, I am personally interested in writing a mod_log_journal for Apache that provides all the kinds of stuff you can get out of mod_log_config, all nicely indexed into different journald fields -- how on Earth is journald expected to be able to provide that to Rsyslog?
Posted Feb 1, 2014 17:07 UTC (Sat)
by dlang (guest, #313)
[Link] (6 responses)
the applications are sending their data to the journal via /dev/log and syslog calls, just exactly the same way that they would send it to syslog if the journal didn't exist (plus capturing stdout and stderr which are usually unformatted garbage)
Posted Feb 2, 2014 0:29 UTC (Sun)
by mchapman (subscriber, #66589)
[Link] (5 responses)
I'm not sure what you mean by "passthrough mode" here. The metadata was never inline with the messages in the first place, even before it got into journald.
Posted Feb 2, 2014 22:35 UTC (Sun)
by dlang (guest, #313)
[Link] (4 responses)
Posted Feb 3, 2014 0:00 UTC (Mon)
by mchapman (subscriber, #66589)
[Link] (3 responses)
I think we might be talking about different things.
I am interested in providing structured messages -- multiple fields containing all sorts of useful metadata -- from applications themselves. I am not too concerned about the "automatic" metadata journald adds; it's clear that if journald can add that, then so can any other logging system.
I note that you've linked to RFC 5424 elsewhere. The structured data extension in that does look interesting; I wonder if journald can support it (or if anybody has asked the developers or proposed it to them).
Posted Feb 3, 2014 4:54 UTC (Mon)
by dlang (guest, #313)
[Link] (2 responses)
The RFC5424 structured data really isn't used very much, and even rsyslog (who's primary developer is the author of RFC5424) didn't have the ability to actually _use_ this data until a few months ago.
in spite of the official standard, the de-facto standard is looking like it's going to end up being JSON structured messages. All of the syslog daemons can parse JSON structured logs and act on the different variables in them (and modify them for that matter)
Posted Feb 3, 2014 18:38 UTC (Mon)
by rodgerd (guest, #58896)
[Link] (1 responses)
Why it seems to have taken systemd to light a fire under the arses of syslog developers is another question entirely...
Posted Feb 3, 2014 18:55 UTC (Mon)
by dlang (guest, #313)
[Link]
It didn't, the syslog developers had already been working on these things (and many others)
RFC5424 and the CEE standardization projects came out several years before the systemd journal.
The only features that the systemd journal had (or at least claimed to have) that syslog didn't have at release were
capturing STDOUT and STDERR (which I'm not sure is really that good an idea, as sloppy as many developers are with these)
trusted properties (some of which are racy to gather, so not as reliable as you may think), which had been discussed and was on the TODO list, but had not made it to the top
log 'security', which as many noted, the journald security did not protect you against malicious root on your system unless you sent data elsewhere, and sending data elsewhere is usually the best defence anyway. Syslog developers have implemented better anti-tampering features that specifically document what external connectivity is needed to make things secure.
It doesn't help that distros tend to ship such old versions of the syslog daemons. It is hard to see progress when the version that's included is several years old.
Posted Feb 1, 2014 18:15 UTC (Sat)
by raven667 (subscriber, #5198)
[Link] (1 responses)
Posted Feb 2, 2014 22:20 UTC (Sun)
by dlang (guest, #313)
[Link]
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time
From anti-systemd to pro-systemd in the shortest time