The Grumpy Editor's guide to surviving the systemd debate [LWN.net]

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 18, 2014 4:03 UTC (Tue) by dlang (guest, #313)
In reply to: The Grumpy Editor's guide to surviving the systemd debate by mchapman
Parent article: The Grumpy Editor's guide to surviving the systemd debate

It does make it so that all log entries after the corruption are lost though.

This is one of the problems with binary logs

to post comments

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 18, 2014 5:04 UTC (Tue) by mchapman (subscriber, #66589) [Link] (3 responses)

Only up until the next rotation... which would presumably happen when you update systemd to fix the bug, if not earlier.

Look, I'm not denying that there are bugs that can corrupt the logs or lose large numbers of log messages. But the original claim -- that corruption yields an "empty" journal that can only be made non-empty through the use of "special commands" -- is patently false.

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 18, 2014 11:59 UTC (Tue) by notninjaz (guest, #99725) [Link] (2 responses)

As I read it, the original "empty" statement was referring to the new journal which would be empty as in a clean slate, comparing it to offering the user a new clean slate filesystem rather than trying to repair a corrupted one.

This thread has brought out some of the tradeoffs, though, such as a recommendation to rotate the log frequently to minimize the impact of possible corruption. For the servers I administer, I would prefer logging to a text file and risking loss of a fraction of a second of logs if an entry is corrupt instead of rotating a journal hourly and potentially losing 59 minutes should there be corruption.

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 18, 2014 13:14 UTC (Tue) by mchapman (subscriber, #66589) [Link] (1 responses)

> As I read it, the original "empty" statement was referring to the new journal which would be empty as in a clean slate, comparing it to offering the user a new clean slate filesystem rather than trying to repair a corrupted one.

Maybe that was what it was referring to. At any rate, it's just not true. When a journal file is rotated (whether that be because journald has detected corruption in it, or because it's reached its maximum size, or because the admin has simply asked for it by sending SIGUSR2 to the journald process), its contents are still read automatically by journalctl when searching and iterating through the logs.

The journal is not a single file; it is the sum total of *all* journal files. As far as I know, the only way to create an empty journal is to literally remove all of these journal files.

The Grumpy Editor's guide to surviving the systemd debate

Posted Nov 18, 2014 14:51 UTC (Tue) by notninjaz (guest, #99725) [Link]

Right, I was referencing the statement: "Yes, it's possible some header corruption could mean an entire file is unreadable. I haven't personally seen that, but I can imagine that it's a concern for some people. You can limit the damage by ensuring the files get rotated frequently. There's {System,RunTime}MaxFileSize config options available; perhaps corresponding {System,RunTime}MaxRotateInterval might be useful. Worst comes to worst, you could just SIGUSR2 the journald process regularly from Cron or a timer unit."

I meant to express a design preference rather to critique any particular software with regard to text vs. binary logs.