The journald design is horrible to the point of useless

Posted Dec 2, 2011 14:18 UTC (Fri) by dlang (guest, #313)
In reply to: The journald design is horrible to the point of useless by anselm
Parent article: That newfangled Journal thing

When I talk about rsyslog running on Solaris, I'm not talking opensolaris. as for the other flavors of Unix, if any users of them care enough about rsyslog or syslog-ng running on that platform it will happen (either from contributed code or from paying for support). But in any case the existing syslog daemons do just fine receiving messages from those systems (and various other proprietary systems like Cisco) and then handling those logs just like all the other logs. This means that you can combine them in whatever log store you choose to use. You won't get things like trusted attributes from such systems, but that's because those systems don't offer that capability, and journald sure isn't going to solve this since it won't run on those systems (and can't make such systems provide info that isn't there in the first place)

finding which file in a series of time-rotated files a particular time is in is pretty trivial, as is finding the time in the log file once you have opened it (at least if you use a normal test editor), you can either do the binary search yourself by jumping to various line numbers, or just search for the timestamp you are looking for. You are really overstating the difficulty here. Yes this can be done wrong (rolling the logs daily, even if you have gigs of logs in a day is usually not a wise thing to do for example)

the idea that you can get the log messages when you ask for the status of a program only works in the trivial case where all the logs are written by the one pid that was started by the program you are asking about the status. If that pid started other processes that then wrote logs, systemd (or equivalent) isn't going to have a way of knowing for sure which 'service' those log messages are for.

I don't believe that the Journal is going to enforce programs writing well structured logs any more than the windows event log does (see Rainer's comment that in spite of the 'structure' being enforced by windows event log, the problem of analyzing logs on windows is the same mess that it is on *nix)

It sounds as if you have decided that anything that LP writes is the Pony that you want and any criticism of it just means the person doing the criticizing is against all progress.

The journald design is horrible to the point of useless

Posted Dec 2, 2011 17:07 UTC (Fri) by raven667 (subscriber, #5198) [Link] (1 responses)

If that pid started other processes that then wrote logs, systemd (or equivalent) isn't going to have a way of knowing for sure which 'service' those log messages are for.

I think this particular statement is probably not true. systemd uses the cgroups feature to associate all processes as part of a generic "service" or "session" which is not fooled by daemonization so it could reliably associate all logs across multiple PIDs with a particular "service".

The journald design is horrible to the point of useless

Posted Dec 2, 2011 19:05 UTC (Fri) by dlang (guest, #313) [Link]

this is a point I didn't consider, but identifying what cgroup something is in is going to be another racy lookup isn't it?

in any case, I've suggested to Rainer that he add this lookup to the trusted properties that rsyslog provides. Once that is there then it will be trivial to split the logs by service (and therefor trivial to do a tail of the most recent logs from any service)

The journald design is horrible to the point of useless

Posted Dec 2, 2011 17:16 UTC (Fri) by anselm (subscriber, #2796) [Link]

the idea that you can get the log messages when you ask for the status of a program only works in the trivial case where all the logs are written by the one pid that was started by the program you are asking about the status.

AFAIK this is wrong. Systemd puts each service into its own control group. Processes that the service generates stay in the same control group. The system status command can then consider all messages from the same control group, or the same systemd service, in order to catch everything from the service process and its children. This is actually in the proposal.

That this is a lot more difficult to do with the current infrastructure (SysV init and (r)syslogd) doesn't mean that it is, in fact, impossible in general. Systemd is not a complete exercise in futility.

It sounds as if you have decided that anything that LP writes is the Pony that you want and any criticism of it just means the person doing the criticizing is against all progress.

No. As far as I am concerned it is absolutely OK to criticise Lennart's and Kay's proposal, just as it is absolutely OK to criticise the existing syslog infrastructure and protocol. Progress results not from hanging on to the existing stuff at all costs, but from critical evaluation of both the old and any newly proposed stuff, and from putting the best ideas of both together to create something that is better than what we had before. Whether that is, in the end, a beefed-up syslogd or something entirely different is irrelevant as long as it solves the problems at hand and there is a reasonable compatibility path to the existing infrastructure. You will note that this also seems to be Rainer's attitude, although he is (for understandable reasons) somewhat biased towards the »beefed-up syslogd« end of the spectrum of possible solutions. (Which is of course OK.)

I've been using various syslogd implementations for a very long time and have usually been able to get them to do what I need, but that doesn't mean I'm blind to possible improvements that may come in from elsewhere. This IMHO is a better approach than dismissing all of a number of possible improvements outright just because one does not like the people who came up with them. No one (not even Lennart or Kay) believes that the journald proposal solves all possible problems with logging, but pretending that there are no problems with logging at all, or no serious problems, or that if there are in fact problems then the journald proposal doesn't solve them either, doesn't actually lead to progress.

The journald design is horrible to the point of useless

Posted Dec 2, 2011 17:28 UTC (Fri) by raven667 (subscriber, #5198) [Link] (3 responses)

It sounds as if you have decided that anything that LP writes is the Pony that you want and any criticism of it just means the person doing the criticizing is against all progress.

I can't speak for the OP but from from my perspective, as someone who is not personally invested in the outcome, about 80% of the comments seem to be baseless negative personal animosity against LP or an appeal to tradition against progress. Half of the other 20% are making actual technical arguments pointing out flaws in the proposal and the other half are defending LP, pointing out positives about the proposal or just advocating keeping an open mind. Those that are defending tend to be responding more the 80% than to the actual constructive criticism of the 10%

Some of the constructive criticism is very pursuasive and I'm not at all sure that the journal is the right way to go, unlike with systemd which was so obviously the right, UNIX, way to do things that I wish it was written decades ago. I can't help of think about daemontools supervise and multilog, which I used very successfully for many years, and see LP as this generation's DJB.

The journald design is horrible to the point of useless

Posted Dec 2, 2011 19:06 UTC (Fri) by dlang (guest, #313) [Link] (2 responses)

> and see LP as this generation's DJB

I think that may people would agree with you, including a LOT of the people who are being critical of LP and this proposal.

there are good reasons (and not just personal animosity or his license choice) that DJB's software did not take over the world.

LP: the new DJB?

Posted Dec 2, 2011 22:50 UTC (Fri) by anselm (subscriber, #2796) [Link] (1 responses)

there are good reasons (and not just personal animosity or his license choice) that DJB's software did not take over the world.

However, some of DJB's ideas did become popular on technical merit. Maildir comes to mind.

I don't think Lennart and Kay are doing badly in comparison at all. Systemd in particular is likely to go a lot farther than Qmail or DJBDNS ever did, simply because, whatever its detractors may say, it does have all kinds of advantages compared to SysV init (the situation is a lot more obvious than with, say, Qmail vs. other MTAs) and the two don't go for DJB-style rest-of-the-world-be-damned my-way-or-the-highway backwards incompatibility in quite the same way. After all, systemd still interfaces with SysV init scripts and traditional syslogd but in a manner that introduces interesting and useful new features.

As far as journald is concerned, we'll have to see; maybe the future will just be a closer association between systemd and rsyslogd, which wouldn't be a bad thing either.

LP: the new DJB?

Posted Dec 3, 2011 3:39 UTC (Sat) by raven667 (subscriber, #5198) [Link]

Actually when I read some of the first design docs for systems I was greatly reminded of daemontools which I have used in all my big, 24/7 systems over the last decade. I chose daemontools because of its technical merits, because of obvious deficiencies in sysv init and despite the fact the software comes from Mars with its own filesystem layout and other needless complications. We even tried using svscan for pid 1. Systemd has all the technical merits and none of the deficiencies.

I hope this journal thing leads to some improvements but I'm less certain that the journal as described is the be all and end all of logging. Let's see where it goes.