|
|
Log in / Subscribe / Register

Thank you...

Thank you...

Posted Nov 21, 2011 19:23 UTC (Mon) by cmccabe (guest, #60281)
In reply to: Thank you... by niner
Parent article: That newfangled Journal thing

A lot of people have compared using the journald binary format to using gzip to compress your logfiles. However, this comparison may be misleading.

gzip is a block compressor. You don't have to have access to the entire file to recover some of the data. You can actually seek to the beginning of a gzip block and start reading without having read all the data that came before.

A lot of media formats have this property as well-- mp3 for one. If the beginning of the mp3 is lost, you can still access the last part. Video formats often have 'key frames' that can be used to recover synchronization if the decoding got screwed up.

Being able to recover something, even if part of the file is messed up, is a very important property for log files. Disks can and do fail and start returning garbage-- I just had one turn this week. I hope that the journald designers think about issues like this, at least in later versions of the format.


to post comments

Framing with FEC.

Posted Nov 21, 2011 19:43 UTC (Mon) by ejr (subscriber, #51652) [Link]

Indeed. A simple framing including some forward error correction code around some well-known, extensible encoding (maybe text, maybe protocol buffers, dunno) could prove even more resilient to failures than the plain-text logs. Same can be said for the text files used by dpkg, etc. VMS FTW?


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds