LWN.net Logo

systemd v38 released

From:  Lennart Poettering <lennart-AT-poettering.net>
To:  systemd Mailing List <systemd-devel-AT-lists.freedesktop.org>
Subject:  [ANNOUNCE] systemd v38
Date:  Wed, 11 Jan 2012 05:19:53 +0100
Message-ID:  <20120111041953.GA3414@tango.0pointer.de>
Archive-link:  Article, Thread

Hey,

http://www.freedesktop.org/software/systemd/systemd-38.ta...

This is the first release with the journal. The journal replaces both
systemd-kmsg-syslogd and systemd-stdout-kmsg-bridge, thus simplifying
greatly our set of always-running services. The journal is quite
complete at this time, but a small number of bigger features are still
missing. Documentation is currently terse and will be extended in the
coming versions. If you want to see the effect of the journal, try
"systemctl status" which is now hooked up with the journal and will show
the most recent log output of a service.

Compatibility with classic syslog implementations is provided, via a
socket /run/systemd/journal/syslog, to which all messages are forwarded,
regardless whether they came in via /dev/log, the journal native
protocol or any other source. To make your syslog implementaiton work
with this make sure that it binds to that socket instead of /dev/log
which is now systemd-journal property.

Note that the journal will store log data only on /run/log/journal,
which makes it volatile. If a subdirectory /var/log/journal is created
the journal will instead store its data on /var, thus making the journal
persistant.

systemd-journalctl may be used to explore the journal. By default it
generates an output that is a pixel perfect copy of the usual
/var/log/messages log format. It can alternatively generate a couple of
differnt outputs, controllable with "-o verbose", "-o short-monotonic"
or "-o json". Use "-f" for something that resembles "tail -f".

This release is more kind of a test release, i.e. like you'd place it in
your distribution's equivalent of Rawhide, not the kind you'd update
your released distro with. We hope to stabilize things in the next
coming releases a bit.

Bill Nottingham (1):
      Allow 'list-unit-files' to run with --root.

Dan Walsh (1):
      label: fix labeling of symbolic links

Daniel Walsh (1):
      namespace: remount namespace root dir for SLAVE to avoid propagation of mounts from the
namespace to the host

Dave Reisner (1):
      systemctl-completion: always invoke with --no-legend

Dexter Morgan (1):
      Add Mageia support

Gregs Gregs (1):
      fsck: Fix typo in comment

Jonathan Nieder (1):
      audit: do not complain if kernel lacks audit

Kay Sievers (33):
      update TODO
      journal: silence gcc warnings
      Makefile.am: consistently use tabs
      logind: add 'login' subdir to include dirs
      logind: fix POTFILES location
      build-sys: restructure logind parts in Makefile.am and add --disable-logind
      timedated: move sources to subdirectory
      localed: move sources to subdirectory
      hostnamed: move sources to subdirectory
      hostnamed, localed: move config files to subdir
      binfmt: move sources to subdirectory
      journal: restructure Makefile.am
      libsystemd-id128: restructure Makefile.am
      pam_systemd: restructure Makefile.am and move inside ENABLE_LOGIND
      libsystemd-daemon: restructure Makefile.am
      build-sys: remove redundant _CFLAGS assignment
      build-sys: restructure sd-daemon, sd-readahead, sd-login files
      build-sys: more remaining parts of sd-deamon to their own section in Makefile.am
      gnome-ask-password-agent: require libnotify >= 0.7.0
      build-sys: login - remove specified include dir; all subdirs are in the list now
      build-sys: switch to tar xz
      do not install drop-in files; the shared library should be used by default
      build-sys: rename 'rootdir' to 'rootprefix' like udev and kmod uses
      conf-parser: silence compiler warning
      logind: add needed include for sd_notify()
      build-sys: switch autoconf's default prefix from /usr/local to /usr
      build-sys: install pam module in $rootlibdir
      build-sys: add some headers to fix 'make distcheck'
      build-sys: use ${ac_default_prefix}
      build-sys: add headers to EXTRA_DIST to fix 'make distcheck'
      autogen: add --with-rootlibdir= and switch /usr/local to /usr
      fix compiler warning
      autogen.sh: diet and sync with udev and kmod

Lennart Poettering (184):
      journal: add preliminary incomplete implementation
      journal: store XOR combination of entry data object hashes to identify hash lines
      journal: replace linked list by hashmap when merging files
      journal: split user logs into their own journal files
      journal: implement parallel traversal in client
      util: properly detect what the last capability is
      Merge branch 'master' into journal
      journal: only fallocate() what we really need to avoid slowness on file systems which do not
support fallocate natively
      util: fix build
      journal: place machin journals in machine specific subdirectories
      journal: add cgroup path to entries
      journal: add cgroup path to entries
      journalctl: find journal files in right path
      journal: implementation rotation
      journal: implement parallel file traversal
      journal: fix field retrieval by name
      journal: synchronize seqnum across files
      journal: when the same entry is in two files, skip over them in sync
      journal: move field index from file into journal object
      journal: if two entries match with everything but seqnums, they are still identical
      journal: properly implement matching with multiple matches
      journal: unify code for up and for down traversal
      journal: implement seek to head/tail
      journal: automaticall reset location when the set of matches changes
      update TODO
      README: explain dependency on cgroups
      plymouth: fix ply proto endianess issues
      initctl: don't use dbus connection after PID 1 got respawned
      cgroup: always recreate cgroup before we try to apply attributes
      mount: order remote mounts after both network.target and remote-fs-pre.target
      units: drop [Install] section from remote-fs-pre.target
      update TODO
      Merge remote-tracking branch 'zbigniew/systemadm_changes'
      cgroup: immediately remove all cgroups which run empty
      update TODO
      macro: fix ALIGN_TO macro definition
      journal: implement multiple field matches
      TODO
      Merge branch 'master' into journal
      update TODO
      journal: add native protocol to journald, and client side API to send journal messages
      journal: enforce limits on open journal files
      journal: introduce mandatory sd_journal_printf() priority parameter
      journald: filter fields send from client starting with underscore
      man: document the sd-login interfaces
      sd-daemon: fix #include lines since we now ship a shared library
      man: build new man pages
      man: sd_readahead is not actually available in libsystemd-daemon
      build-sys: add rules for man page aliases
      man: add sd-login(7) page
      man: various updates
      man: extend sd-login(7) in regards to mixing D-Bus and synchronous library calls a bit
      hashmap: add hashmap_first_key()
      man: generate HTML instead of XHTML with XSL docbook to work around 'fsfunc' noise
      man: switch to UTF-8 output, to work around charset issues
      journal: implement inotify-based live logging logic
      journal: fix matches
      journal: fix space reservation limit enforcement
      journal: add inline compression support with XZ
      journalctl: add json, export, short and verbose output modes
      journalctl: add command line parsing
      journal: add missing compress.[ch]
      journal: properly handle first inline bisect array entry
      journald: enforce some syntax restrictions on field names sent from the client side
      journal: implement stdout transport
      journald: implement sophisticated rate limiting
      udev: exclude loopback device from udev rule based sysctl application, since we can just
apply that directly at boot
      journal: fix typo
      journal: fix hash table lookup logic
      journald: when checking available disk space for rate limiting, cache the results
temporarily
      journal: never mmap beyond file size
      journald: increase rate limit burst rate
      journald: flush /run to /var as soon as it becomes available
      journald: don't recheck /var availability more often than 30s
      remount-api-vfs: handle another OOM condition
      journald: add missing header
      journal: add unit files and shared library glue
      journald: forward all syslog messages to syslogd
      journal: fix a few bad memory accesses and leaks
      journal: disable default debug logging
      journal: move max_use into metrics structure
      Merge branch 'journal'
      socket: rename the PassCred= option to PassCredentials=, since we don't want to needlessly
abbreviate options unless they are very well established
      kmsg: drop unused variable
      journal: automatically deduce journal metrics from file system sizes
      git: update gitignore
      logind: move logind into its own subdirectory
      logind: move more files into subdirectory
      journal: move symver file into subdirectory
      util: fix warning
      login: move libsystemd-login.pc.in into src/login/
      test: rename test directory
      login: move the PAM module to src/login/ since it is just a client to logind
      build-sys: fix mageia kbd mappings
      build-sys: make readahead and vconsole optional
      login: move systemd-user-sessions.service into login/, too
      cryptsetup: split off cryptsetup into its own subdir
      build-sys: move kbd-model-map to locale/
      build-sys: make quotacheck and randomseed optional
      socket: only add dependency on kmsg socket to socket units which have any kind of exec
program specified
      readahead: bring export definition of sd-readahead in line with sd-daemon
      nspawn: get rid of BUFFER_SIZE, use LINE_MAX instead
      login: introduce sd_pid_get_service()
      journal: add _public_ to all sd-journal calls, and add parameter checks
      sd-id128: add _public_ to all exports, and add validity checks for all parameters
      journald: add _SYSTEMD_SESSION, _SYSTEMD_OWNER_UID, _SYSTEMD_SERVICE to all entries
      journald: treat a read-only /var identical to an unmounted one
      journal: never fail if we cannot access /var, just print a warning
      systemctl: hook up systemctl with the journal
      logind: if we can't open /dev/tty0, assume there is no VT subsystem and don't pretend we
could do VT switching
      logind: don't watch vcsa if nobody cares
      logind: send out Lock signal when locking
      journald: fix _SYSTEMD_CGROUP= values
      sd128: add format string macros for outputting sd_id128_t variables
      sd128: rename SD_FORMAT_ID128_STR to SD_ID128_FORMAT_STR to follow naming scheme
      journalctl: add -n switch
      journal: add missing error check
      journal: add memory barrier before linking in newly created entries
      journal: apply seek check before resizing window, and refresh stat data if necessary
      journal: when increasing window, make sure to use the increased window
      journald: don't rotate on startup
      journal: don't mind too much if we can't find a monotonic timestamp
      journal: add missing error check
      journal: fix reverse traversing of entries
      journalctl: fix counting of -n parameter
      journalctl: only output 10 most recent lines in --follow mode
      journald: store _SYSTEMD_UNIT= instead of _SYSTEMD_SERVICE= field, since processes might also
be related to mount, swap or socket units, not just services
      systemctl: shortcut log output for non-service, non-socket, non-mount, non-swap units
      journal: it's not a problem if the realtime jumps, hence don't ensure monotonicity of
realtime for entries we write
      journal: beef up journal output of systemctl and journalctl
      journal: fix XZ build flags
      journald: parse configuration file
      util: when printing status updates during boot, take terminal width into account
      log: minor optimization
      journald: add configuration file options to forward all logged data to kmsg, console, syslog
      util: never ellipsize welcome message
      build-sys: move public header files into a dir of their own
      journal: add missing sd-messages.h
      journalctl: add --new-id switch to generate a new 128Bit id
      git: only ignore toplevel systemd file
      journal: fix include in sd-journal.h
      build-sys: add stub makefiles to subdirs
      build-sys: drop colorgcc
      headers: fix git URLs for source files
      README: correct license claims
      journalctl: rework short output mode to rebuild full syslog message
      journal: move sockets into their own subdir
      journal: store used transport in journal
      journal: store used transport in journal
      execute: talk directly to the journald, instead to the stdout-syslog-bridge
      sd-journal: close reading side of sd_journal_stream_fd() file descriptors
      journal: rename priority_prefix to level_prefix, since that's how we call it in PID 1
      journal: rename syslog tag to identifier since that's what we call it on the server side.
      login: implement sd_session_get_service()
      stdout: remove stdout-syslog-bridge since it is now obsoleted by journald
      journal: introduce log target 'journal' for executed processes
      special: fix name of journal socket unit
      kmsg-syslogd: remove kmsg-syslogd, since it's entirely obsoleted and replaced by journald
      journald: start the journal after the syslog socket, so that the syslog socket queues syslog
messages from early boot on
      units: remove left-over unit file
      service: add dependencies on configured sockets
      unit: properly update references to units which are merged
      journald: shorten the time we wait for a forwarding syslog
      journald: start journald right away, don't wait until activation
      journald: introduce systemd_journald.forward_to_kmsg=1 (and friends) to enable kmsg
forwarding globally via kernel cmdline
      main: fix spelling
      load-fragment: fix parsing of Socket= setting
      build-sys: move .pc files next to the matching sources
      journalctl: rename --new-id to --new-id128 in order not to introduce yet another new name
      sd-id128: let's make our API a bit smaller, since sd_id128_make_v4_uuid() is dispensable
      journald: remove inner loop debug message
      journald: add kmsg source
      journalctl: add new short-monotonic output mode
      journalctl: display source timestamp, not journald timestamp, if known
      shutdown: exclude processes with argv[0][0] from killing
      update TODO
      service: brutally slaughter processes that are running in the cgroup when we enter START_PRE
and START
      shutdown: add link to root storage daemon text
      unit: implement new PropagateReloadTo=/PropagateReloadFrom= operations
      Update TODO
      journalctl: always show monotonic timestamp even if it's from an old boot
      systemd: reconnect to syslog as soon as the journal is fully up
      sd-journal: implement a number of non-implemented calls from the API for now
      journal: don't realign window twice
      build-sys: bump version
      build-sys: fix upload rule for xz

Michael Biebl (2):
      build-sys: fix typo in help string
      build-sys: link systemctl and systemd-journalctl against libsystemd-id128

Michal Schmidt (55):
      manager: fix a crash in isolating
      utmp: remove unneded parameters
      utmp: no need to zero a struct before overwriting it with memcpy
      utmp: initialize store with the found entry, not with the lookup key
      utmp: for DEAD_PROCESS write the current time to wtmp
      man: fix a typo in signal number
      units: drop unnecessary 'StandardOutput=syslog'
      units/fedora: let rc-local.service log to syslog
      units/suse: let rc-local.service log to syslog
      service: don't warn if the pidfile still exists after SIGCHLD
      job: colored status messages on boot
      man: fix typo in sd_notify
      execute: avoid logging to closed fds
      execute: make setup_pam() return -errno when possible
      execute: log errors from "sd(EXEC)"
      pam-module: use the correct session type "unspecified"
      pam-module: treat "cron" in PAM_TTY as empty tty
      let mount and swap units log to the configured defaults
      socket: add option for SO_PASSCRED
      shutdownd: use PassCred=yes in the socket unit
      syslog: use PassCred=yes for the /dev/log socket
      man: document the PassCred option
      add a generator to pull rc-local.service in
      rc-local: no need to check if the script is executable
      rc-local: order after network.target
      util: fix error checking after fgets()
      path: use %m instead of strerror(errno)
      path: refactor PathSpec usage
      path: add PathModified (= PathChanged + IN_MODIFY)
      service: handle services with racy daemonization gracefully
      service: stop the service if ExecStartPost ends with a failure
      unit: garbage collect units with load error
      systemctl: print 'error' load state in red
      path: add missing pieces for PathModified
      unit: fix false positive in check for unneeded unit
      unit: check for unneeded dependencies even when unit stop was expected
      pam-module: add a couple of debugging prints
      systemctl: fix typo in 'is-enabled'
      tmpfiles: use an enum instead of plain char for item type
      tmpfiles: rename a couple of functions
      tmpfiles: use a common function to set owner/group/mode/label
      tmpfiles: separate a generic item glob processing function
      tmpfiles: add RECURSIVE_RELABEL_PATH ('Z')
      man: document 'Z' in tmpfiles
      man: mention that 'Z' ignores uid/gid/mode
      service: use 'syslog+console' for sysv_console
      tmpfiles: apply chown, chmod for 'Z' entries too
      tmpfiles: add 'z', like 'Z' but not recursive
      man: fix misplaced remark in description of Sockets=
      execute: fix losing of start timestamps
      dbus: register to DBus asynchronously
      dbus: no sync D-Bus connection flushing
      log: never block on syslog in PID 1
      util: fix switching to console unicode mode
      util: switch the console to text mode on reset

Michał Górny (2):
      man: Fix out-of-source manpage builds.
      build-sys: Fix missing directories with parallel 'make install'.

Ran Benita (3):
      bash-completion: update with new verbs and arguments
      bash-completion: add completions for systemd-loginctl
      bash-completion: rename file since it is no longer for systemctl only

Thomas Jarosch (2):
      condition: Fix file descriptor leak in test_capability()
      Fix same expression on both sides of '&&'

Tim Waugh (1):
      '@' is an 'ampersat' not an 'ampersand'; let's call it 'at symbol'

Tollef Fog Heen (1):
      service: Drop rcN.d runlevels from SysV services that also exist in rcS.d

Tom Gundersen (3):
      random-seed: convert poolsize from bits to bytes
      cryptsetup-generator: avoid ordering cycle on swap
      man: fix SEE ALSO in hostname(5)

Zbigniew Jędrzejewski-Szmek (16):
      systemadm: break timestamp formatting out into a seperate function
      systemadm: allow sorting of jobs and units
      systemadm: split the type+status combo box into type combo & status checkbox
      systemadm: filter on swaps, paths, and timers too.
      systemadm: add a wrappable label and use it for status lines
      systemadm: add libgee as dependency and use it for a unit map
      systemadm: display dependencies sorted
      systemadm: use color for dependency links
      systemadm: use bold for "requires", etc.
      systemadm: make the dependency listing  selectable
      systemadm: catch exceptions generated by dbus
      systemadm: coalesce id and decription fields
      systemadm: adjust row numbers after removing 'aliases'
      systemadm: use colors for id too, remove color from fragment link
      systemctl: make list-unit-files output more economical
      fix compilation error with 'PathSpec redefined'

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
_______________________________________________
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


(Log in to post comments)

systemd v38 released

Posted Jan 11, 2012 17:19 UTC (Wed) by kragilkragil2 (guest, #76172) [Link]

Systemd and journal are probably nice, but from where I am standing pulse audio is degrading. On my F15 netbook it starts to crackle (pkill pulseaudio and a app restart fixes it). There are bugs in the tracker but nobody seems to care.
If this is the future of systemd and journal we are in for a rough ride.

systemd v38 released

Posted Jan 11, 2012 18:07 UTC (Wed) by acobar (guest, #74253) [Link]

For some strange reason, it started to happen on my openSUSE 11.4 too. I am not totally sure if this is caused by pulse audio but the problems started after I installed more components with "*pulse*" when trying to install clementine-player (it happens also on amarok now). I am still pinpointing the source of the problem. Anyway, I really had lots of trouble with it on past. On paper it is a wonderful solution, in practice .. well, lets just say that it is still troublesome. Really hope the same does not happens with systemd, even though I can see some similarities.

systemd v38 released

Posted Jan 11, 2012 18:26 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Check volume in pavucontrol. Looks like you're experiencing clipping because of volume boost.

systemd v38 released

Posted Jan 11, 2012 22:27 UTC (Wed) by drag (subscriber, #31333) [Link]

If the audio works, but it's crackling then that is probably a issue.

If you are getting blips and squeeks and perfectly fine videos won't run well (jerky, freezes, high cpu usage, etc) then that is usually when you have problems with the drivers or pa doing things incorrectly. That was a normal problem to run into with just Alsa before Pulse came along and forced a lot of fixes on the drivers.

Pulseaudio should be smart enough to know that if you drive a particular card near or at 100% volume then it will cause clipping and adjust the "max output" appropriately. Isn't there some tweak database or something that keeps track of weirdness with specific models?

Although the problem could be in your speaker's amplifier and not have anything to do with the PC side of things.

Not sure.

systemd v38 released

Posted Jan 12, 2012 0:35 UTC (Thu) by dmarti (subscriber, #11625) [Link]

If it always does it for you, though, your bug report is like pure gold. Please post your hardware info and the combination of applications that causes the audio trouble to your distribution's bug tracker, and someone should be able to track it down, propagate it upstream, or whatever. Generalized complaints without bug reports are not so helpful.

crackle

Posted Jan 12, 2012 1:58 UTC (Thu) by tialaramex (subscriber, #21167) [Link]

IMO Audio clipping is really unlikely to sound like "crackle". If you've got a keen ear, mild digital clipping from e.g. playing two things at once can be audible, but it won't sound like crackle.

The way people tend to get the idea that it might is they play with some algorithm left over from a crappy 8-bit computer which can't afford to do saturating addition, let along floating point maths, and instead of digital clipping they get wrapping (ie +0.6 added to +0.6 gives -0.2 instead of +1.0). Wrapping certainly can cause a crackling sound. Nothing in the modern Linux audio pathways should do wrapping though.

If you exceed the actual performance of the analogue outputs (e.g. you try to push too much signal through your headphones) you'll get distortion, but again that wouldn't sound like crackling, and it'd tend to give itself away by only occurring during loud passages of music or sound effects.

However crackling sounds definitely /can/ come from tiny underuns. The Thinkpad laptops for a while all seemed to have some chip which couldn't (be bothered to) set the register bits which show how much data is left in the buffer on the sound chip. If the driver failed to workaround/ compensate then you would get underuns that sound like crackling.

Underuns can also happen from a too busy PCI bus (unlikely on a modern machine where the busiest peripherals have dedicated PCIe channels) or if your software is sufficiently far from realtime that it sometimes misses the deadline to supply new buffers of audio. But in the latter case they'd tend to be longer underuns, which again don't sound much like crackle.

Last of all of course, it could actually be a good old fashioned analogue crackle, caused by a loose cable, a deteriorating solder joint etc.

Not at all about systemd v38 being released :)

Posted Jan 12, 2012 12:32 UTC (Thu) by Ford_Prefect (subscriber, #36934) [Link]

Bug reports upstream are welcome as well -- https://bugs.freedesktop.org/enter_bug.cgi?product=PulseA... -- you might not always get lightning-quick responses but we do try to help.

systemd v38 released

Posted Jan 12, 2012 8:55 UTC (Thu) by ibukanov (subscriber, #3942) [Link]

On at least two systems the audio has degraded for me with kernel updates. For example, with initial Fedora 15 kernel everything was nice, but kernel updates either degraded the quality or just stopped audio working. So I would not blame the pulse. Still I share your experience that audio-related bugs gets significantly less attention than bugs filed for other areas.

systemd v38 released

Posted Jan 15, 2012 4:17 UTC (Sun) by sbergman27 (guest, #10767) [Link]

I *always* have some problem or another with Pulseaudio. It's almost never needed. So these days, after doing enough testing to satisfy myself that PA is, yet again, not working right, I yank it out by the roots, pulling all the unneeded components for all the individual programs with it, and then sound works perfectly on every system I've done this on.

Used to be, PA advocates would blame my non-Fedora distro for just doing it wrong. But my current desktop machine is CentOS 6.2. You know. The direct clone of the premier distro of the Red Hat family, where they take the time to do it right. (3.75 years of development, in this case, plus 1.25 years of real-world bug fixing.) Unsurprisingly, sound didn't work at all for Totem.

I yanked all the unnecessary PA stuff and sound immediately began working perfectly.

In fairness to CentOS/RHEL I should say that PA and Package Kit are the *only* noticeably buggy apps I've found in the whole distro. And Package Kit's bugs are really only an annoyance. Not crippling for a home desktop, like PA. I very much would not want to disparage this excellent distro.

systemd v38 released

Posted Jan 11, 2012 18:27 UTC (Wed) by markatto (subscriber, #70420) [Link]

I am still unconvinced that a binary log format is a good idea. I am all for a structured format, but why not something like JSON or YAML?

systemd v38 released

Posted Jan 11, 2012 18:38 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Because binary formats are much faster to parse/write?

Pixel-perfect emulation of syslogd is a nice thing. The only aspect of Journal that I hate is its use of UUIDs for message IDs. I'd very much prefer reverse domain names.

systemd v38 released

Posted Jan 11, 2012 18:51 UTC (Wed) by theophrastus (guest, #80847) [Link]

yet-more proof by blatant assertion:
UUID formats are much faster to parse/write (than reverse domain names).

and/or:
have minor differences in the speed of parsing and writing log entries proven a significant burden to the "typical use situation" in the past?

systemd v38 released

Posted Jan 11, 2012 18:58 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

>yet-more proof by blatant assertion:
>UUID formats are much faster to parse/write (than reverse domain names).
Not really. UUIDs/reverse domains are not parsed at all, they are used as opaque markers. So the difference in cost of parsing is really minimal.

>have minor differences in the speed of parsing and writing log entries proven a significant burden to the "typical use situation" in the past?

Actually, yes. Also, the issue of compactness led to use of gzip-compressed logs which are binary _and_ hard to use with standard Linux tools (like awk/sed).

systemd v38 released

Posted Jan 11, 2012 19:15 UTC (Wed) by theophrastus (guest, #80847) [Link]

You're supporting the change to systemd/journal by appealing to the use of "standard Linux tools (like awk/sed)"?

And shouldn't we both avoid weasel terms like "minimal", "minor", "faster", when neither one of us has the statistics to back up these assertions? (so i'm taking my own advice and shutting up until i'm sure there's significance)

systemd v38 released

Posted Jan 11, 2012 19:18 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Yes, I've been trying git versions of systemd. I really like ability to do easy queries by date/time and then pipe results to awk/sed.

Much easier than the classic way of sifting through "debug"/"debug.1"/"debug.2.gz"/"debug.3.gz", especially when interesting events are split between two log files (mighty annoying).

systemd v38 released

Posted Jan 12, 2012 0:55 UTC (Thu) by Wol (guest, #4433) [Link]

Not to throw oil on the flames or anything :-)

But didn't Cyberax express a *personal* dislike of UUIDs? And then you jumped on him for claiming that reverse domains were better. What's wrong with expressing a dislike?

And while I think Cyberax may be wrong (what happens if you want multiple IDs for a single domain?), I've often objected to things on the basis of a gut feel only to discover that there's a reason behind it once I've investigated. Not always valid, but usually worth investigating.

Cheers,
Wol

systemd v38 released

Posted Jan 12, 2012 18:36 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

>And while I think Cyberax may be wrong (what happens if you want multiple IDs for a single domain?)
Simple, you do "org.mysoftware.FileEvent", "org.mysoftware.NetEvent", ....

systemd v38 released

Posted Jan 11, 2012 19:25 UTC (Wed) by Eckhart (guest, #74500) [Link]

> Not really. UUIDs/reverse domains are not parsed at all, they are used as opaque markers. So the difference in cost of parsing is really minimal.
If two UUIDs are not equal, this inequality can be checked way more efficient than with reverse domains.
For example, take the following names
org.freedesktop.NetworkManager.ConnectionLost
org.freedesktop.NetworkManager.ModemAdded
(which I just made up).
You have to compare 12 bytes just to verify those two are not equal.
In contrast, take these two UUIDs I just randomly generated:
21e0777f-c6c2-4033-8dcd-2824e7a14a9a
c4b952a1-5a59-4ff8-89bc-1554856c98d4
They already differ on the first byte.
In general, given two random UUIDs, a negative comparison needs to check 1.0039216 bytes (1+1/256+1/256^2…)

It's even worse then this

Posted Jan 11, 2012 20:26 UTC (Wed) by khim (subscriber, #9252) [Link]

You can compare UUIDs using two instruction in a tight loop (VPCMPEQB then CMPB). This means that "really minimal" difference in parsing speed may be 3x or 5x - depending on what you need to do.

systemd v38 released

Posted Jan 11, 2012 20:34 UTC (Wed) by ballombe (subscriber, #9523) [Link]

Just start by the end when comparing reverse domains name!

Beside, the speed for comparing UUID is 1+1/16+1/16^2 since only 16 characters are used.
For reverse domain name it depends on the probability distribution of letters, but assuming the one in
http://en.wikipedia.org/wiki/Letter_frequency
we get about the same thing (1+1/15.26+…)

systemd v38 released

Posted Jan 11, 2012 21:03 UTC (Wed) by nybble41 (subscriber, #55106) [Link]

> Beside, the speed for comparing UUID is 1+1/16+1/16^2 since only 16 characters are used.

I think the assumption was that UUIDs would be stored and compared in their binary form, not as strings.

systemd v38 released

Posted Jan 11, 2012 21:34 UTC (Wed) by Eckhart (guest, #74500) [Link]

> Just start by the end when comparing reverse domains name!

> Beside, the speed for comparing UUID is 1+1/16+1/16^2 since only 16 characters are used.

As has been pointed out, UUIDs are normally stored in their binary form.

> For reverse domain name it depends on the probability distribution of letters, but assuming the one in http://en.wikipedia.org/wiki/Letter_frequency we get about the same thing (1+1/15.26+…)
I'm not sure how you deduce that number from the Wikipedia article. Using the letter frequency is not sufficient alone, since the letter-frequency of the nth-last letter in a word is not the the same as the general letter frequency. For example, 'y' is pretty common as a last letter in a word in English language, but almost non-existant as a second-to-last letter (the only word i could come up with is 'dye').

Also, reverse string comparison depends on how you store them. If they are just C-style strings, reverse comparison is the worst thing you could do.
UUIDs on the other hand are of fixed length (16 bytes). Databases love that.

systemd v38 released

Posted Jan 11, 2012 21:24 UTC (Wed) by aleXXX (subscriber, #2742) [Link]

I'll always prefer seeing "org.freedesktop.NetworkManager.ConnectionLost" over "c4b952a1-5a59-4ff8-89bc-1554856c98d4".

I don't think I have to explain why.

Alex

systemd v38 released

Posted Jan 11, 2012 21:41 UTC (Wed) by Eckhart (guest, #74500) [Link]

> I'll always prefer seeing "org.freedesktop.NetworkManager.ConnectionLost" over "c4b952a1-5a59-4ff8-89bc-1554856c98d4".

You forgot that there's also a plain-text description of the error. The UUID is there to provide semantical information about the error.

systemd v38 released

Posted Jan 12, 2012 13:15 UTC (Thu) by dsommers (subscriber, #55274) [Link]

But when analysing logs, you don't have to look up that "c4b952a1-5a59-4ff8-89bc-1554856c98d4" means "org.freedesktop.NetworkManager.ConnectionLost" ... think grep log files

Having a human understandable "index" instead of a non-human-understandable index may help finding what you're looking for in logs, especially when looking for trends in certain error situations. Something sysadmins often do.

systemd v38 released

Posted Jan 12, 2012 15:00 UTC (Thu) by lethargo (subscriber, #26367) [Link]

Agreed. When scanning visually, it's easier to pick out lines with "org.freedesktop.NetworkManager.ConnectionLost" mixed with other reverse domain names than it is to pick out EITHER a GUID or the full-text description.

Also, if you have two references to the "key", e.g. one in the log and one in an internet posting or a script someone wrote to check for it, as a human it is easier to read the reverse domain name to double-check that it is the same than it would be to read a GUID. (And a script might not have the full text message.)

systemd v38 released

Posted Jan 12, 2012 22:31 UTC (Thu) by marm (guest, #53705) [Link]

All this holds only if the only thing you want to do with the identifiers is comparing them for equality. Reverse domain names have a huge advantage of being hierarchical, so you can easily filter for all messages generated by a given subsystem or sub-subsystem etc.

systemd v38 released

Posted Jan 13, 2012 2:49 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

Well,

1) It's even easier to check strings for inequality if one stores string length. Most likely different strings would have different length. Besides, it's possible to store strings in reverse order.

2) The difference between comparing UUIDs and strings is not even close to significant on most sane log sizes. And insane-sized logs most likely need special instruments anyway.

3) Domain names are structured, so I can easily query things like "all events of org.freedesktop.powermon". Not so easy with UUIDs.

4) UUIDs are not even guaranteed to be unique as they are supplied by untrusted applications.

And last but not least, domain names are human readable while UUIDs are most definitely not.

systemd v38 released

Posted Jan 13, 2012 15:19 UTC (Fri) by nix (subscriber, #2304) [Link]

Given that the entire purpose of these UUIDs is to allow people to google for other people experiencing similar failures, I suggest you're overthinking things. Comparison is irrelevant, because only rare search operations will need to compare anything: structure is irrelevant, because these UUIDs are not intended as the *only* identifier of a message, nor a means of imposing structure upon it, just a random string which is often correlated with the appearance of that message, and a human-readable component will still exist: and even uniqueness does not matter, because humans will be looking at the output and are very good at discarding irrelevant hits. (In any case, duplication of UUIDs seems less likely than people misunderstanding the scheme and regenerating UUIDs when the message has not changed much.)

systemd v38 released

Posted Jan 13, 2012 15:40 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

I understand. But why not make the message IDs a little bit more useful?

I'm not seeing _any_ downsides to reverse domain names compared to UUIDs. They are just as googleable and easier to use.

systemd v38 released

Posted Jan 11, 2012 19:20 UTC (Wed) by dskoll (subscriber, #1630) [Link]

I am convinced that binary-formatted logs are a bad idea. But worse than that: The Journal's binary format is not officially documented and is subject to change at any time, turning it from a bad idea to a diabolically horrible idea.

systemd v38 released

Posted Jan 11, 2012 19:51 UTC (Wed) by Eckhart (guest, #74500) [Link]

> The Journal's binary format is not officially documented and is subject to change at any time
Here, fixed it for you: "The Journal's binary format is not officially documented yet and is subject to change for the time being"

systemd v38 released

Posted Jan 11, 2012 21:35 UTC (Wed) by dskoll (subscriber, #1630) [Link]

When does "the time being" end?

systemd v38 released

Posted Jan 11, 2012 21:42 UTC (Wed) by Eckhart (guest, #74500) [Link]

> When does "the time being" end?
It might have something to do with "We hope to stabilize things in the next coming releases a bit."

systemd v38 released

Posted Jan 12, 2012 1:43 UTC (Thu) by dskoll (subscriber, #1630) [Link]

Well, OK. When the format stabilizes, it will improve from a disastrously horrible idea to a merely bad idea.

systemd v38 released

Posted Jan 11, 2012 20:00 UTC (Wed) by gioele (subscriber, #61675) [Link]

> Note that v38 is a test release;

Can we please go back to sane versioning conventions where release numbers are not just incremental? "v38" tells me little about the fact that this release is not safe for production; "v38-beta1" screams "do not install me on your box!".

systemd v38 released

Posted Jan 11, 2012 22:35 UTC (Wed) by zdzichu (subscriber, #17118) [Link]

Something as core as init shouldn't be installed by Joe Random user. Distro integrators for core components are supposed to read mainlinglist and announcements.

Version naming conventions

Posted Jan 12, 2012 13:30 UTC (Thu) by rvfh (subscriber, #31018) [Link]

So who's going to test it then? Distros, on the 10 PCs they have lying around? DO you expect admins to start using once the format is frozen? What if they find that something crucial to them has been overlooked?

Alpha, beta and rc are here to let the user get an idea of the risk they take, and I'd appreciate it everybody was using them, or a similar scheme like 1.4.90...

Version naming conventions

Posted Jan 12, 2012 14:22 UTC (Thu) by Eckhart (guest, #74500) [Link]

> So who's going to test it then? Distros, on the 10 PCs they have lying around? DO you expect admins to start using once the format is frozen? What if they find that something crucial to them has been overlooked?

In short: people that are interested in trying out the new features, and are experienced enough to either compile from source or use a compiled version from e.g. Fedora Rawhide, Debian Experimental etc. Because they are interested, they tend to read the release announcement.
Also, people have different ideas of what "beta" actually refers to. If you think of "beta" as "medium probability of killing kittens", then v38 is definitely not beta. The "beta" in v38 is more a "not all announced features are implemented yet, don't be disappointed", combined with a "speed can probably be improved", at least that was my impression.

systemd v38 released

Posted Jan 12, 2012 0:12 UTC (Thu) by drag (subscriber, #31333) [Link]

I know that when running:
systemctl status blah.service

And be able to get the log files immediately that way if the service fails would certainly make things go a bit quicker.

I envision it having a way to automatically have it spit out the logs for the service since it was last started. Something like that.

systemd v38 released

Posted Jan 12, 2012 0:45 UTC (Thu) by fuhchee (subscriber, #40059) [Link]

"The journal is quite complete at this time, but a small number of bigger features are still missing."

One cannot write such an oxymoron without tungue in cheek, surely.

Copyright © 2012, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds