Enabling the persistent journal in Debian

It seems unlikely that anyone on any "side" of the systemd war that has raged in Debian over the last few years thought that the results of the recent general resolution (GR) vote ended the matter. The vote showed a clear preference for moving ahead with systemd as the preferred init system, though it was far from any kind of landslide—there were definitely plenty of voters who would have preferred a different outcome. It was a complicated GR, with a wide spectrum of options, but at this point, the project as a whole has spoken. Actually implementing some of the changes that the GR enabled may not have the smooth path that some might have hoped for, however.

On February 1, Michael Biebl posted a message to the debian-devel mailing list noting that he had fixed a wishlist bug (from 2013) by enabling the systemd persistent journal. Prior to that, journald would log to the non-persistent /run/log/journal directory by default and rsyslog would create the persistent text log files in /var/log. The change to the Debian systemd package would create the /var/log/journal directory where journald will store its persistent binary log files. That way, the journals will still be available after a reboot.

The message said that new installs and upgrades of the systemd package would create the directory, but it also included instructions on how to revert to the existing behavior; further upgrades to the systemd package will respect that choice. Beyond that, though, running both the persistent journal and rsyslog means that the log files are effectively stored twice on disk, so Biebl may ask the Debian ftp-masters to lower the priority of rsyslog so that it is not installed by default for the upcoming Debian 11 ("bullseye") release. Those who want to have a different system logger can add it after the initial install, of course.

Steve McIntyre replied that the change to the systemd package was fine for new installs, but that he would rather not see upgrades of the package add the persistent journal. "Those people with existing logging setups will be surprised by this." Didier "OdyX" Raboud disagreed but suggested that the change be prominently announced. Biebl thought that it could be surprising not to add the functionality on an upgrade:

Moreover, there is some relevant history from a Debian downstream that has already made the switch to the persistent journal. Biebl asked Dimitri John Ledkov to comment on how the transition went for Ubuntu, which enabled the persistent journal on installation or upgrade in late 2017. Ledkov said that the process went quite well overall, with few real problems. A large chunk of the Ubuntu user base now has the change since "systems installed with / upgraded to 18.04 LTS overwhelmingly have persistent journal enabled by default across the board". Biebl agreed with Raboud that documenting the change is important, so he filed a bug to that effect.

rsyslog default

The possible switch away from rsyslog by default concerned Dmitry Smirnov, however. "I think that replacing rsyslog with journald is two steps back in regards to functionality and flexibility." He explained that rsyslog has more features than journald and is more mature, thus more stable. He felt that changing the default system logger "yields no benefits to say the least".

Russ Allbery agreed with Smirnov's thoughts about rsyslog, but pointed out that the change would just be for the system default.

There are some minor benefits to the switch as well, Allbery said:

Smirnov was unconvinced that "those benefits are enough to justify replacing a very solid and reliable logging system", however. But, as Allbery said, journald is already upstream of rsyslog today, so the default is to run two logging daemons, which seems somewhat suboptimal; he is not sure that it makes sense to keep doing so, especially with the addition of the persistent journal. He did acknowledge that the switch might require some relearning:

There was some additional grumbling about switching to the journalctl command to look at log entries, which some see as a regression in functionality; it is certainly different than using familiar tools on text log files for those who do not change the default, but only if rsyslog is dropped from the default install. The rsyslog option is not going away, as Allbery pointed out. In addition:

GR consequences

Inevitably, some complaints about the GR outcome and what it means going forward were aired. The result of the vote may not have necessarily dictated the change in default for Debian as a whole, at least directly, but dropping the priority of rsyslog was only mentioned as a possibility. Meanwhile, Debian project leader Sam Hartman said that the outcome of the vote has already had a positive impact on the community:

But a suggestion that systemd opponents leave Debian en masse over the results of the GR was met with a number of responses, some perhaps less well-considered than others. Hartman's reply was typically even-keeled; he had previously acknowledged the unhappiness and frustration the GR results might engender in some, but the call to leave seemingly went too far. "It sounds like you were suggesting that people should leave as a way of pressuring or punishing the project." That is not the kind of community that Hartman is hoping and aspiring for; there are options for those who are unhappy with the outcome:

That message is worth reading in its entirety. Hartman empathizes with those who are on the "outside", even though he has determined that systemd is the best choice—for now. He simply asks that those opposed respect the other project members and the Debian project itself. "If you don't try to burn the house down on the way out, we'll be happy to see you again later if your needs or Debian change."

The problems with Debian and systemd have been going on for quite some time now; it's been almost exactly six years since the Debian technical committee voted to make systemd the default for the "jessie" release. That did not really end the matter, obviously, and the recent GR outcome will not either—at least completely. The trend, however, looks like things are moving in a direction where the rift is starting to heal—or opponents are moving on. Both are good for the Debian community as a whole. Enabling the persistent journal is a pretty minor change in the grand scheme of things; overall, it was met with constructive criticism or agreement. One hopes that whatever other systemd-derived changes are coming over the next months or years can garner a similar response—perhaps even without the need to re-litigate the "systemd question" yet another time.