That newfangled Journal thing

Posted Nov 30, 2011 7:34 UTC (Wed) by alison (subscriber, #63752)
In reply to: That newfangled Journal thing by dlang
Parent article: That newfangled Journal thing

dlang offers:
The hashes would be trivial to add (and for that matter, you could use a database store and have it do the normalization and hashing as part of the insert today)

Why not in fact use git (or similar) to frequently snapshot a flat ASCII log file, storing the deltas in a repo over which hashes are generated in the usual manner? The UUIDs need not be present in the ASCII log but can be git tags that are stored as part of the commit data. Then git can be used in its usual fashion to propagate the log to networked machines if desired. Furthermore, the generation of hashes and remote propagation provide the usual level of verifiability we associate with git.

In other words, why not keep the flat file, but generate a verifiable log in a standard machine-readable format from it on the fly? Is this not how we are already using DVCS to generate trees from programmer-generated source code? Why wouldn't git work just as well with programmer-formatted log messages?

to post comments

That newfangled Journal thing

Posted Nov 30, 2011 8:49 UTC (Wed) by dlang (guest, #313) [Link] (5 responses)

git is horrible overkill and inefficient if all you are doing is dealing with one log file.

tripwire, ossec, and other similar tools already can track a log file and detect the difference between the file being extended and the file being modified.

Also, if you have another machine you can send data to, just send the logs using the standard syslog mechanisms. unless you are generating gigantic amounts (hundreds of thousands of logs per second) of logs or would be logging over a wan, the bandwidth needed for the logs is just not going to be significant

That newfangled Journal thing

Posted Nov 30, 2011 22:20 UTC (Wed) by elanthis (guest, #6227) [Link] (4 responses)

I love it. Bigotry and emotions run rampant in technical discussions more and more these days.

Argument: journald is more complex than syslogd, and complexity is evil!

Proposal: use the simpler syslogd and then add complex and error-prone log parsing toolkits to get 70% of the features of journald at 200% the complexity cost.

Rationale: Pulseaudio was buggy a couple years ago.

That newfangled Journal thing

Posted Nov 30, 2011 23:26 UTC (Wed) by dlang (guest, #313) [Link] (3 responses)

for most people, the new features of journald won't matter, they either don't have any worm device to store the hashes to to make things secure, ot they don't care about such features because they send all of the logs to a remote system.

he's 'solving' a problem that isn't really there, and he doesn't actually solve the stated problem.

Also, please point out anywhere that I have said anything about pulseaudio.

That newfangled Journal thing

Posted Nov 30, 2011 23:34 UTC (Wed) by anselm (subscriber, #2796) [Link] (2 responses)

Read the <expletive> proposal. Journald isn't just about the hashes.

That newfangled Journal thing

Posted Nov 30, 2011 23:44 UTC (Wed) by dlang (guest, #313) [Link] (1 responses)

I have read the proposal, and I still think that overall he is solving problems that don't exist in ways that don't really solve the stated problem

if you want to ignore the hashes part of things, we can talk about the structured log part of things. logs are only as structured as the programmer creating them makes them, if you have a super-detailed log structure available and the programmer creates a field "details" type "string" and puts everything into that field it is going to be just as unstructured as syslog traditionally has been.

He ignores or is ignorant of recent standards in syslog (some of which go back quite a few years)

he has a few new ideas buried in the proposal, but they are so overshadowed by misstatements and 'solutions' to problems that already have documented, standardized solutions (that are not compatible with the proposed solution other than running logging services in parallel) that it undermines the entire proposal

That newfangled Journal thing

Posted Dec 1, 2011 17:03 UTC (Thu) by kh (guest, #19413) [Link]

Thank you for being a voice of reason.