rsyslog 7.4 released

[Posted June 6, 2013 by jake]

Version 7.4 of the rsyslog system logger has been released. This is the first version of the new 7.4 stable branch and it joins version 7.2.7 as supported versions of the tool. New headline features include support for the systemd journal (both as input and output) along with log file encryption, signatures, and anonymization.

rsyslog 7.4 released

Posted Jun 7, 2013 2:06 UTC (Fri) by yodermk (subscriber, #3803) [Link] (7 responses)

I've been playing with its log normalization features, with export to ElasticSearch. Neat stuff, but very cryptic to use. I've written a tutorial but it is not yet published.

rsyslog 7.4 released

Posted Jun 7, 2013 23:20 UTC (Fri) by treed (guest, #11432) [Link] (6 responses)

Would love to see a tutorial on this. Where would I find it when you finally do publish it?

rsyslog 7.4 released

Posted Jun 8, 2013 11:33 UTC (Sat) by yodermk (subscriber, #3803) [Link] (5 responses)

Trying to get it on my company's devops blog. Failing that I'll just put it on my personal site if I get permission (was written on company time). I'll come back here when there's a link.

rsyslog 7.4 released

Posted Jun 11, 2013 23:42 UTC (Tue) by treed (guest, #11432) [Link] (4 responses)

Where is your company's blog? I don't know who you work for.

And where is your personal blog? I don't know who you are. :)

Also: I was reading about rsyslog and signed logs here:

http://blog.gerhards.net/2013/05/rsyslogs-first-signature...

Do you know if GuardTime is some third party service provider of some sort from whom I have to license this functionality? I'm having trouble imagining what a "signature provider" is or why it is tied to a particular company.

rsyslog 7.4 released

Posted Jun 12, 2013 3:20 UTC (Wed) by raven667 (subscriber, #5198) [Link] (1 responses)

> Where is your company's blog? I don't know who you work for.
> And where is your personal blog? I don't know who you are. :)

It's interesting how much information can be found just by googling someone's userid because especially in the tech community they tend to be persistent for a long time, people sometimes forget all the places they use the same handle and how much is public.

rsyslog 7.4 released

Posted Jun 13, 2013 2:52 UTC (Thu) by treed (guest, #11432) [Link]

I did search for him but I searched for his username plus rsyslog and didn't find much useful. Apparently my search was too specific because just his username turned up much more. Thanks.

rsyslog 7.4 released

Posted Jun 12, 2013 10:04 UTC (Wed) by rgerhards (guest, #70316) [Link]

A rsyslog "signature provider" is actually just a small piece of software. As your posting made me aware that this will probably become a frequent question, I blogged about it:

http://blog.gerhards.net/2013/06/what-is-rsyslog-signatur...

The posting also contains some answers to question I guess some folks may have in regard to the actual guardtime provider and which I didn't answer in the initial (long) post about it.

rsyslog 7.4 released

Posted Aug 6, 2013 3:27 UTC (Tue) by yodermk (subscriber, #3803) [Link]

Sorry for the delay but it's finally been published:

http://developer.rackspace.com/blog/rsyslog-and-elasticse...

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 14, 2013 20:30 UTC (Fri) by dlang (guest, #313) [Link] (9 responses)

per this buzilla report
https://bugzilla.redhat.com/show_bug.cgi?id=974132

Fedora 19 and earlier have a bug in systemd that causes it to write corrupted binary log files. Any program trying to read those files ends up with an endless loop of messages.

Since rsyslog 7.4 adds the imjournal module to pull data from the journal, you end up with rsyslog eating 100% cpu as it reads the looped messages from the journal and outputs them per it's config.

Apparently there is an updates systemd version that prevents this from happening in the future and they are trying to make the upgrade process detect and do something to fix the problem in existing journal logs (delete the offending logfile?>)

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 14, 2013 20:36 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (7 responses)

an update for systemd that fixes this problem has already been pushed

https://admin.fedoraproject.org/updates/systemd-204-8.fc19

Just FYI, this won't affect anyone not running Rawhide or Fedora 19 pre-release since rsyslog 7.4 won't be pushed as an update for the older stable releases (Fedora 18 and 17).

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 14, 2013 20:49 UTC (Fri) by dlang (guest, #313) [Link] (6 responses)

unless those people run into some other reason to upgrade to rsyslog 7.4 or newer. It's not uncommon for people to want newer features on something developing as rapidly as rsyslog is and therefor need to upgrade

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 14, 2013 20:58 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (5 responses)

I really doubt that. Logging software isn't something you typically upgrade rapidly on production systems especially without testing it thoroughly. In any case, if one really wants it, go ahead and get both the rsyslog and systemd updates together.

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 15, 2013 3:18 UTC (Sat) by dlang (guest, #313) [Link] (4 responses)

The vast majority of people don't upgrade at all (well, the vast majority don't even realize it's there :-)

But when you start doing complex things with the logs, it's not uncommon to bump into a reason to upgrade, if for no other reason than you ask for help with something and the normal reaction of "are you on a current version" comes back.

It's nowhere close to being a large percentage of users or machines, but it's far from uncommon.

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 15, 2013 3:25 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link] (3 responses)

It really depends. If you are running software custom compiled or built directly from upstream and you ask upstream to help you out or if you are reporting a bug to a rapidly updating community distribution like Fedora, it is not uncommon for maintainers to check in with the latest version in part because they are fairly close to it anyway and it is easy enough to get and test. However if you are running a long term enterprise style distribution, maintainers generally would not ask you to do that.

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 15, 2013 4:12 UTC (Sat) by dlang (guest, #313) [Link] (2 responses)

not that this is that important (I mostly just posted this issue so that people would be aware of it in case they were testing, not because I thought it's a big deal)

But as someone who spends a significant amount of time answering rsyslog questions, there are a lot of people running RedHat (and the ancient versions provided with it) that ask the upstream mailing list directly. It's entirely possible that there are even more questions that RedHat is fielding, but quite a few are bypassing them.

As I say, not really important

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 15, 2013 4:36 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link] (1 responses)

That is interesting if they are really RHEL customers are not CentOS users. Red Hat is certainly fielding a lot of logging questions but if a subset is paying for support and not taking advantage of it, I do wonder why.

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 15, 2013 7:44 UTC (Sat) by dlang (guest, #313) [Link]

well, since some of them have said that they think that they can't upgrade to a newer version without loosing support, I'm pretty sure a fair number of them are really running RHEL, not CentOS.

it doesn't help than RHEL 5 is still using rsyslog 3.x by default (although with 5.9 or so I heard they added a 5.x package as an option), even RHEL 6 is only shipping 5.x

they aren't the only distros way behind. Ubuntu 13.04 is still on 5.x for example.

bug in systemd interacts badly with rsyslog 7.4

Posted Jun 15, 2013 11:29 UTC (Sat) by rgerhards (guest, #70316) [Link]

some more info: the problem surfaced after the 7.4.0 Fedora package replaced imuxsock by imjournal, obviously in a try to get hold of information hidden by the journal. As long as you continue to use imuxsock, no problem happens. I guess rsyslog's imjournal was the first piece of software that actually depended on the journal being corruption free (I read inside the bug tracker that this seems to be a rather old journal problem).

In any case, I have just made some quick adjustments to imjournal that will help mitigate such issues in the future:

https://github.com/rgerhards/rsyslog/commit/9057b1ccfb0a7...

Please note that the rsyslog project does recommend using imuxsock, except in cases where the journal hidden data is vitally important. Thankfully, I made a presentation on this a couple of days ago ;)

https://www.youtube.com/watch?v=GTS7EuSdFKE

(around minute 7 is the recommendation) Looks like I was good in predicting...