| |||||||||||||
The logger meets linux-kernelThe logger meets linux-kernelPosted Jan 5, 2012 0:08 UTC (Thu) by cmccabe (guest, #60281)In reply to: The logger meets linux-kernel by neilbrown Parent article: The logger meets linux-kernel
I wrote a logging system that was designed to be truly "free."
See https://github.com/cmccabe/redfish/blob/master/util/fast_... Also the .h, etc.
In order to make it truly fast, it had to be lockless (at least most of the time), thread-local (no cacheline ping-pong) and entirely in userspace.
I don't see how anything that involves a syscall can ever truly be "free." Even a system which just involves an mmap'ed ring buffer protected by a single mutex will tend to sharply limit scalability on the multi-core hardware of the future (and present).
Maybe Journald can accumulate a bunch of messages and send them to the kernel all at once. However, that begs the question: what happens to unflushed messages on a crash? They could use a signal handler to take care of this problem, but a lot of programs handle signals themselves.
(Log in to post comments)
The logger meets linux-kernel Posted Jan 5, 2012 0:59 UTC (Thu) by neilbrown (subscriber, #359) [Link]
This is why I think it would be much more effective to focus on implementing a higher-level API.
Android can implement it with a trivial library that writes out to their in-kernel ring buffer. You can implement it using your awesome fast_log, and both can still run all the wonderful Android applications, each on the kernel of their choice.
Then when all the app developers start saying to Android "The log system from that McCabe guy is much better than yours - could you start providing that one instead", then we can all forget about the android-logger in the kernel and move on.
The logger meets linux-kernel Posted Jan 5, 2012 1:27 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
just make sure that this higher level ABI doesn't require ioctl calls for implementation.
The logger meets linux-kernel Posted Jan 5, 2012 1:42 UTC (Thu) by cmccabe (guest, #60281) [Link]
Well, my log system doesn't address a lot of the issues that Android's does. For example, handling logs from different processes that don't trust each other.
It seems like the only reason they wanted this in the kernel in the first place was speed. Sending messages to a daemon over a socket or pipe was considered too slow. Unfortunately, they ended up adding a lot of policy like the log stream names "radio", "main", "events", etc. This was also the rationale for putting Binder and friends in kernel space-- to get that nice zero-copy performance along with the security they wanted. Maybe what we really need is a secure circular buffer IPC primitive rather than all these special-purpose widgets.
It's interesting that the Android guys went with unstructured logging. They control the entire userspace, so presumably it would be a lot easier for them to force developers to write structured log messages than it will be for Red Hat to get people to use structured Journal logs. There is also something bizarre about the fact that 1970s-era UNIX systems were fine using unstructured logging, and small cell phone systems are fine using it, but people claim that it is too inefficient to run on servers. However, maybe there's something I'm overlooking and the current focus on structured journald logging will be useful. Time will tell, I guess.
The logger meets linux-kernel Posted Jan 9, 2012 15:07 UTC (Mon) by sorpigal (subscriber, #36106) [Link]
>There is also something bizarre about the fact that 1970s-era UNIX systems were fine using unstructured logging, and small cell phone systems are fine using it, but people claim that it is too inefficient to run on servers. However, maybe there's something I'm overlooking and the current focus on structured journald logging will be useful. Time will tell, I guess.
I chalk it up to simple Windows-envy.
The logger meets linux-kernel Posted Jan 14, 2012 3:40 UTC (Sat) by swetland (subscriber, #63414) [Link]
Performance was definitely one concern, especially on earlier ARM9 devices where context switches involved cache flushes due to virtually tagged caches.
Unstructured (or really a bit semistructured -- timestamps, pid, loglevel are structured, message content is not) logs are partially to avoid having to have knowledge of the precise software running or maintaining an ever-expanding definition of log message formats to interpret the logs from an arbitrary device that's encountered. Very valuable in a large ecosystem with a huge variety of devices, software versions, etc.
The name/sizes stuff should just move to platformdata / module params / or the like. An oversight that's easily enough corrected.
|
|||||||||||||