This is the heart of the problem

Posted Feb 13, 2014 19:46 UTC (Thu) by dlang (guest, #313)
In reply to: This is the heart of the problem by mathstuf
Parent article: The Debian technical committee vote concludes

rsyslog can capture the same things from the kernel that systemd does (it may not capture everything today, but there's no reason it couldn't capture anything that is desired)

rsyslog gathers the early boot logs from the kernel after it starts. the journal would need to do the same thing if it starts at the same time as rsyslog (and if it starts earler to gather more, why don't you start rsyslog earlier as well)

I don't think you have any idea of the amount of filtering that rsyslog can do. I know that LP didn't when the journal was added. color is a display thing, not part of the log. FSS didn't do what it claimed, and rsyslog now has the equivalent that actually works.

there is no need to have journal as a shim between the app and rsyslog other than the fact that the systemd developers mandate it.

rsyslog does implement the published API to the journal. however this does not allow rsyslog to replace the journal, because the internal API to the journal is not stable, running without the journal is 'unsupported'

as far as udev goes, the fact that distros are still using udev without using systemd indicates that your claim that they have to be used together is false. The problem is that the developers have repeatedly opposed the idea of making it possible to build udev without building all of systemd. I don't think people have submitted patches to do so (but I wouldn't be surprised if they had) because they have been so vocal in saying that they would reject any such patches.

This is the heart of the problem

Posted Feb 13, 2014 21:50 UTC (Thu) by paulj (subscriber, #341) [Link] (1 responses)

FSS? That's the new cryptographic protocol to try secure the logs which journald implemented even before it had even been published, never mind peer-reviewed? I note there is at last a paper available for it: http://eprint.iacr.org/2013/397.pdf .

Can you expand on it not doing what was claimed? How serious is the discrepancy?

This is the heart of the problem

Posted Feb 16, 2014 3:20 UTC (Sun) by dlang (guest, #313) [Link]

as many people noted when it was announced, if the logs are stored locally they can be tampered with. you may not be able to modify the file in place, but you can create a complete replacement, with all the signing in place.

It requires interaction with an outside system to make a tamper-proof log, either to send signatures elsewhere or to get a token that cannot be recreated with only data on the local system.

This is the heart of the problem

Posted Feb 13, 2014 21:57 UTC (Thu) by peter-b (guest, #66996) [Link]

> FSS didn't do what it claimed, and rsyslog now has the equivalent that actually works.

Yeah, that'll be a [citation needed], please.

This is the heart of the problem

Posted Feb 14, 2014 9:25 UTC (Fri) by smurf (subscriber, #17840) [Link] (1 responses)

rsyslog starts up much later and is prone to miss some things. All the userspace output from jobs started before rsyslog used to be lost before the journal came along, and some systems spew more kernel logs up to that time than the kernel has buffer space for.

You can tell the thing to be totally transparent if you don't like it.
Though I wonder why you'd want to – "systemctl status FOO" prints the last couple of a job's journal entries. Which is something I never expected I'd *not* want to do without.

Also, I don't want rsyslog to filter output for the simple reason that I won't know whether I'll need it before the fecal matter has actually encountered the rotating air circulation facilitator.

Also², rsyslog is just text, so there's exactly one sort criterion when all is said and done (the file name). The journal has so many ways to filter for the output you're actually interested in it's not funny, shows you possible values for fields …

I do wonder what's going on here. I mean, "the output from daemons gets lost when you use sysv-rc, so that's The Unix Way. systemd captures that stuff and doesn't even use an externel helper for that, so it's not Unix, so it's Monolithic Evil" is not a technical argument, it's religious.

This is the heart of the problem

Posted Feb 16, 2014 3:44 UTC (Sun) by dlang (guest, #313) [Link]

> rsyslog starts up much later and is prone to miss some things.

so start it earlier

>You can tell the thing to be totally transparent if you don't like it.

well, I haven't messed with the journal personally, but the people who have that I trust tell me that it doesn't actually work, and it's still a performance problem.

to be fully transparent, the journal would need to lookup all the trusted properties from SCP_CREDENTIALS, then forge them for the message to rsyslog that's a pretty significant amount of work to do for high log rates.

> Also, I don't want rsyslog to filter output for the simple reason that I won't know whether I'll need it before the fecal matter has actually encountered the rotating air circulation facilitator.

well, that's up to you. you were talking about filtering capabilities, rsyslog can send everything as-is, split it to different destinations, normalize the messages, reformat them, etc.

If you want everything in one place you can do that, if you want things splitup or put into any of a bunch of different types of datastores, you can do that as well.

> Also², rsyslog is just text, so there's exactly one sort criterion when all is said and done (the file name). The journal has so many ways to filter for the output you're actually interested in it's not funny, shows you possible values for fields …

You keep saying that "syslog is text only", what data is it that actually needs to be binary?

rsyslog will deliver the messages to many different things, including the systemd journal if that's where you want it, but it can do that in addition to everything else.

> I do wonder what's going on here. I mean, "the output from daemons gets lost when you use sysv-rc, so that's The Unix Way. systemd captures that stuff and doesn't even use an externel helper for that, so it's not Unix, so it's Monolithic Evil" is not a technical argument, it's religious.

the output from daemons gets lost if you want it to get lost.

random garbage from stderr is not log data, and mostly it's absolutely useless (look at the apache error log for example), having an option to feed it into a logging system is great, pretending that it is all good log data is showing inexperience.

systemd has a lot of things that would be great if they were options to be used when they make sense. But they cause problems when they are made mandatory under all conditions.

There are companies out there that take the attitude that they know better than their customers what the customers want, and they can be very successful if pushing customers to buy what they offer, but that is not what Open Source or Free Software is supposed to be about.

Free and Open Software is supposed to be about empowering the users, and that includes making it possible for the users to do things differently from what the developers imagined. Unless there is a lot of configurablity (like the Linux Kernel has), "Monolithic Evil" is less a religion than it is an observation.