Add sync markers and tags to binary files.
Posted Nov 22, 2011 6:31 UTC (Tue) by eru
In reply to: The Journal - a proposed syslog replacement
Parent article: The Journal - a proposed syslog replacement
This is because text has a record delimiter that is not used within the records (the newline character), making synchronization trivial.
Binary format can easily have the same property: A synchronization marker (not necessarily a single byte) that is guaranteed to not appear in the data. This means the actual data needs some processing to avoid the marker, but this can be cheaper than a conversion to text. Eg. if your sync marker is 0x55, double it if it appears in the payload data. Some other byte combinations starting with 0x55 could tag the type of following data (date, numbers of different sizes, string etc), which also helps parse possibly corrupted files.
to post comments)