That newfangled Journal thing - some thinking required

Posted Nov 21, 2011 19:16 UTC (Mon) by ndye (guest, #9947)
In reply to: That newfangled Journal thing by k8to
Parent article: That newfangled Journal thing

Structured logging is a failure.

There's been so many attempts to build systems with structured logging (sometimes called 'messaging' or 'events') and always the system proves unable to anticipate the needs of real applications, and builds an inflexible overcomplicated system that no one wants.

That's why we still have text-stream-logging to this day.

can be, and has been, said many times:

XYZ has never worked, so it never will

Often over the "40 years" of UNIX, rethinking the fundamentals took place in a proprietary cathedral. The artificial limitation of market share limited the pool of engineers thinking about the problem and its interactions. The non-engineering folks required to think of "market" share so the engineers could eat were also driving anti-engineering efforts to limit the implementation to a segment of brain power, by which I mean at least these two forms of intellect segmentation:

people who, having learned one product, can't cross the chasm of artificial differences to adapt to a new environment (I'm looking at you, AIX)
people who must keep unnecessary differences in their head as they rove between vendors

Because we're in the shared ecospace of Linux, the preponderance of designers and users have opportunity to influence, beginning at design rather than late beta.

Not that I'm advocating this particular reinvention -- I find lots of this distasteful: UUID's, binary encoding, the coupling with systemd (solved?), overloading the "journal" name.

As someone else suggested, maybe Lennart should stop publishing his own thoughts and let someone more diplomatic front his ideas, vetting them for obvious mistakes like the name overloading.

Just in the last month I experienced an outage measured in days instead of hours precisely because of name overloading: It *matters*.

Here's hoping this topic stops reminding me of a maze of twisty passages, nearly alike but not inconsequentially different, strewn with pits, tar-filled and otherwise . . . .

to post comments

That newfangled Journal thing - some thinking required

Posted Nov 24, 2011 8:19 UTC (Thu) by k8to (guest, #15413) [Link]

Yeah, except sometimes it's like poo-pooing people who try to reinvent bicycle wheels. Or people who try to put a CVT on a bicycle, or eliptical gear-wheels. None of them are better at all. In fact the bicycle wheel is one of the strongest per weight human structures ever designed. And it's cheap and easy to fix! But yet some people keep trying to add on carbon struts and such and sell it as better...

In this case, using a structured format means creating code that knows about the structured format. Now you've either got to do something gross like add all the journald interfaces to the kernel, or you have to do something else gross like add a big library between the emitting code and the fwrite() call.

You want to get the logged data out of the process space immediately, and the reason you want to do that is that in the crash scenario, you want to know what was happening at the time of the crash, not 2 seconds earlier when the buffering got last pushed.

Meanwhile, you want other systems to be able to tell what's logged, so you want to be able to watch the datastream as it's landing on disk. With a binary format all of this gets fairly complicated. Everyone has to link in this clunky library, or (worse) go through another daemon so that you have another place to lose buffers when the expected failure happens, to handle which is the first driver for adding logging.

So you have to have a clunky library and it has to be race condition free with its binary format, and it has to be carefully vetted to handle all unexpected binary structures correctly, or else tail -f becomes a security attack surface -- well it does regardless but at least if you carefully vet it the probability of a problem is decreased.

And the benefit of all of this is.. what exactly?