Change... to what?

Posted Nov 20, 2011 23:55 UTC (Sun) by HelloWorld (guest, #56129)
In reply to: Change... to what? by jmorris42
Parent article: That newfangled Journal thing

We appear to be fairly conservative in thinking that after four decades we have found a lot of ideas that have stood the test of time and thus any proposed changes should be viewed with a skeptical eye. This is not a bad thing.

Actually, looking sceptically at things just because they're new is a very bad thing. It's about as resistant to change as it gets.

We have seen what happens when the whole operating system merges into a couple of hopelessly tangled binary chunks that talk among themselves in obscure ways. Just how much of the OS is currently either in or being sucked into the black hole of systemd? Does ANYONE actually understand all of udev/dbus/*kit? Now anyone who didn't have to refer to the source code to figure parts of it out?

People nowadays expect stuff to be integrated; it's not like the components of a modern Linux system talk to each other just for the fun of it. And what is obscure about DBus? One can monitor DBus IPC just fine with tools such as dbus-monitor, one can explore the IPC interfaces offered by the running programs using qdbusviewer. For the first time in the history of Unix, there's an IPC mechanism that works on a higher level of abstraction than Unix sockets and is actually used by significant number of people, and that is one of the reasons why proper integration of the various OS components has even become feasible. Yet, you have nothing else to do that whine about how "obscure" and "tangled" this is.

We believe in small individually replacable parts that do one task well, that are configured and communicate in human readable formats.

Why would I care about the format my programs use to communicate? Those are implementation details. What's would be the point in making DBus IPC unnecessarily inefficient just to make it human-readable, when you'll want to use proper tools to interact with it anyway?

to post comments

Change... to what?

Posted Nov 21, 2011 0:07 UTC (Mon) by dlang (guest, #313) [Link] (30 responses)

> Why would I care about the format my programs use to communicate?

you don't care, until you need to either debug things or add your own component somewhere in the middle, at those points how the components communicate is critical.

Change... to what?

Posted Nov 21, 2011 0:23 UTC (Mon) by HelloWorld (guest, #56129) [Link] (29 responses)

> you don't care, until you need to either debug things or add your own component somewhere in the middle, at those points how the components communicate is critical.
Those are pretty lame excuses. If I want to add my own component I'll use a library anyway, and to debug things one will want to use specialised tools like dbus-monitor anyway.

Change... to what?

Posted Nov 21, 2011 2:42 UTC (Mon) by jmorris42 (guest, #2203) [Link] (28 responses)

> If I want to add my own component I'll use a library anyway, and
> to debug things one will want to use specialised tools like
> dbus-monitor anyway.

In other words, you don't value human readable or everything is a file. An API is all you care about. So what are doing using a system designed around everything you think is stupid? 90% of the world is using systems build around that idea, yet you are trying to bring them to the final few percent of holdouts.

Change... to what?

Posted Nov 21, 2011 4:31 UTC (Mon) by HelloWorld (guest, #56129) [Link] (25 responses)

> In other words, you don't value human readable
In the context of an IPC system, I simply don't care. It doesn't offer any advantages and only makes things slower, so why bother with a human-readable protocol? Can you name even *one* sensible reason?

As for "everything is a file", I didn't comment on that one.

> So what are doing using a system designed around everything you think is stupid?
There's no better alternative in the free software world.

Change... to what?

Posted Nov 21, 2011 4:55 UTC (Mon) by wahern (subscriber, #37304) [Link]

If human readable protocols didn't have value than ASN.1 would have ruled the world. Instead we have a plethora of far more clumsy human readable alternatives.

Change... to what?

Posted Nov 21, 2011 13:35 UTC (Mon) by sorpigal (subscriber, #36106) [Link]

I think this thread is extremely important because it starkly shows just what it is that we're discussing and what's at stake. Some people who don't like Unix and don't believe it's good to be Unix are trying to incrementally make Linux into something very different.

If you advertise up front that you want to use the Linux kernel to build a a system that isn't Unix or Unix-like then perhaps proposals that move things in that direction would not be met with such debate. Instead your entire goal could be examined and we could split into the camp which wants that and the camp which doesn't and everyone could move on.

Not advertising up front what the real goal is makes all discussion after that meaningless.

Change... to what?

Posted Nov 21, 2011 18:38 UTC (Mon) by jmorris42 (guest, #2203) [Link] (22 responses)

> There's no better alternative in the free software world.

So why aren't you trying to change that? GNU has the goal of remaking a Free UNIX. ReactOS has the goal of remaking a Free Windows. You could also be making a designed from the ground up OS with your ideas, you could even base it on the Linux kernel to leverage its wide hardware support. See Android for just how different a userspace can be and run on the Linux kernel.

Seems to me that if you disagree with pretty much all of the underlying design decisions of UNIX that beginning with a base more to your liking would be more productive that trying to remake Linux/GNU/X into some alien abortion.

It is about time for a fork; to drive the heretics from the temple. To drive a stake in the ground and say here we take a stand; NO MORE. The Windows refugees who want 'a Windows that doesn't suck.' can go forth and do that. Elsewhere. If that means we never get 'the year of Linux on the Desktop.' we have to live with that consequence. We have made it twenty years waiting for that miracle and managed to thrive, if it never comes it isn't a terrible thing. Maybe UNIX just isn't for everyone, but for those who DO love it we would kinda like to keep it something we can recognize.

And people call Lannart arrogant???

Posted Nov 21, 2011 19:21 UTC (Mon) by khim (subscriber, #9252) [Link] (11 responses)

Wow. Just... wow. You really, seriously, think that you have the right to drive the heretics from the temple?

This is free world and free software. If you don't want to use Lennart's creations and you don't want to pay for his work - you are free to do so (albeit I doubt you pay RedHat for anything anyway).

But to say that someone should stop doing what they are doing just because they crossed some imaginable line... wow. This is arrogance beyond belief.

If you want to keep UNIX culture and UNIX-like distribution (or distributions) alive - you are equally free to do so. But please stop proclaiming that you somehow have right to send people elsewhere. RedHat will deicide if it wants to continue to support Lennart's work. People will deicide if they want to support RedHat which supports Lennart. But if you want to drive a stake in the ground then you should find a place where people will allow you to do that.

Slackware, Debian, whatever. Find the place where people will be happy to live in the past and will be Ok with supporting "the old ways of doing things". If the froth in this thread is any indication you should be easily able to find supporters and live "happily ever after". But why people who don't share you obsession with "Unix way" should suddenly "move away" id they are Ok with RedHat, Lennart, etc?

Actually I suspect I know why all the froth happens: you understand that in reality "Unix way lovers" is very small (if vocal) group and without people who are doing the real work you'll soon have a system which is only usable on a small subset of hardware and with pretty limited choice of software. Well... this is the price you should pay if you really want to fight for your ideals.

And people call Lannart arrogant???

Posted Nov 21, 2011 22:40 UTC (Mon) by nix (subscriber, #2304) [Link]

You really, seriously, think that you have the right to drive the heretics from the temple?

Nah, he can just lock 'em out with SELinux :)

And people call Lannart arrogant???

Posted Nov 22, 2011 1:19 UTC (Tue) by jmorris42 (guest, #2203) [Link] (2 responses)

> Slackware, Debian, whatever.

Yes. That is exactly what I'm asking. At this point though it is very hard to figure out which, if any, of the major distributions plan on remaining recognizable as UNIX descendants. Which makes it kinda hard to pick one. What I'm saying is it is time for distributions to be honest with us poor users and tell us what the hell they are doing. Are they chasing The Year of Linux on the Desktop to the exclusion of the users who have been and currently ARE using their system? Have they even thought about it or are they just caught up in the daily grind of packaging up the same tarballs the other distos are so they show the same version number on distrowatch?

Ubuntu has pretty much told us they are. And we will probably know for sure about RedHat if they actually ship the controversial bits of Fedora as RHEL 7.0. (And if the paying customers eat the dog food, guess it was a good business decision and best 'o luck to em.) But that leaves questions. What of Debian? Do they just pull whatever upline throws them and follow along or what? Does this go unopposed until even Slackware is unrecognizable? It is clear the Windows refugees know exactly what they want and have the numbers to impose their will upon us, so if we don't speak up, and pretty quickly, I suspect there won't be many options other than *BSD, and that their taunt will have come true. That BSD is for people who like UNIX and Linux is for those who don't like Windows.

Maybe we end up having to port BSD to the Linux kernel... Sort of the reverse of Debian/FreeBSD.

And people call Lannart arrogant???

Posted Nov 22, 2011 2:40 UTC (Tue) by mpr22 (subscriber, #60784) [Link]

As far as I know, Slackware has explicitly said "no" to systemd.

And people call Lannart arrogant???

Posted Nov 22, 2011 6:06 UTC (Tue) by jrn (subscriber, #64214) [Link]

> What of Debian? Do they just pull whatever upline throws them and follow along or what?

You can see an approximate preview of the next Debian release by looking at what "unstable" does today (just like you can see a preview of the next Fedora release by running rawhide). If you don't like what you see, you can even participate to improve it by sending clear bug reports and patches.

Suppose you disagree at a fundamental level with an operating system developer --- for example, let's say you love KDE 3 and are upset that the KDE maintainers moved on to KDE 4. Even then, as long as you are doing the work, you can add packages that work the way you want in parallel (for example, you could package Trinity). If your use case requires changing someone else's package in a way that the maintainer does not want, then the technical committee[1] may listen to your arguments, weigh them (which involves, implicitly, weighing the impact on users and the amount of work you have done to bring your proposal about), and come up with a way to make it happen without sacrificing too much.

I imagine most distros work somewhat like this.

[1] http://www.debian.org/devel/tech-ctte

And people call Lannart arrogant???

Posted Nov 23, 2011 23:12 UTC (Wed) by dlang (guest, #313) [Link] (6 responses)

> Wow. Just... wow. You really, seriously, think that you have the right to drive the heretics from the temple?

why not? Lennert thinks that he does, the only difference is the definition of 'heretic'

And people call Lannart arrogant???

Posted Nov 24, 2011 8:44 UTC (Thu) by anselm (subscriber, #2796) [Link] (1 responses)

The difference is that Lennart's (I think you should take the trouble to spell his name correctly, the guy deserves at least that much respect) approach is backwards-compatible with syslog it is fairly trivial to gate syslog messages into »the Journal« while the Unix traditionalist approach is basically to deny innovation in the syslog space.

It is apparently acceptable to tweak the actual implementation of the syslog daemon, but it would be just as well to acknowledge that the underlying protocol sucks and could use improvement.

And people call Lannart arrogant???

Posted Dec 1, 2011 13:02 UTC (Thu) by nye (subscriber, #51576) [Link]

>the Unix traditionalist approach is basically to deny innovation in the syslog space

That really isn't true. The people I've seen in this thread who are unhappy with journald haven't been saying that everything should stay as it is; they've been saying that evolutionary changes would be better than revolutionary changes, and providing several reasons for that which, on the face of it, seem sane.

Saying "we'd like to improve this carefully rather than starting from scratch" is a far cry from 'denying innovation'.

Well, there are big difference...

Posted Nov 24, 2011 10:32 UTC (Thu) by khim (subscriber, #9252) [Link] (3 responses)

why not? Lennert thinks that he does, the only difference is the definition of 'heretic'

No. The main difference is that Lennart has the mandate to introduce this (and other) changes. Fedora has well-defined procedure where such changes are discussed and accepted and rejected - and so far it was pretty sympathetic to the Lennert's creations. They were considered on their technical merits and accepted or pushed away till they'll be ready. They also have the option to reject proposed changes completely (even it was never implemented so far).

This is normal technical process: we all want to make things better but of course different people differ in the definition of "better". What jmorris42 offers is totally different: we should create some kind of sacred tome which will define out religion and reject everything which does not conform to it. No matter if it's good or bad - if it's sacred tome it's wrong and not allowed.

If you want to worship someone or something - it's your choice, but please do it in your sect, in your temple, don't try to push this on other people.

I understand why jmorris42 does not like to do it this way: world is changing. Just because you refuse to change with it does not mean it'll comply with your wishes and will keep the option to use old code open: *BSD fell in this trap - they are mostly unusable today not because they have some inborn weaknesses but because companies decided they don't care and so it's hard to find hardware to run them on!

It's interesting that Linux itself is in danger of going the same way in some cases: people from Palm complain that when they contact subcontractors and ask about Linux support they are redirected to people who ask which version of Android they plan to use!

But if you fear that this will be the result then you have only two choices actually:
1. Accept changes and "go with the flow".
2. Roll up your sleeves and try to keep your "UNIX-style system" alive (like *BSD people are doing).

Screams "heretics, heretics" are not really a good answer because Linux world has no bishops which can hear you and if someone will try to instill such institution I doubt a lot of people will accept that.

Well, there are big difference...

Posted Nov 24, 2011 11:26 UTC (Thu) by dlang (guest, #313) [Link] (2 responses)

you define heretic as anyone unwilling to accept the latest proposal

others define heretic as people eager to change the system in ways that are not incramental changes, but are instead "throw out what's there and write a replacement, the replacement doesn't need to do everything the old system did because the people who used those features aren't important"

changes that are proposed that are really optional (and not just "pick a different distro if you disagree" optional), and can be ignored if the admin wants to get relatively little opposition (they may get some grumbling if it takes work to figure out how to turn off the new features, but nowhere near the opposition that these re-writes from scratch get)

Well, there are big difference...

Posted Nov 24, 2011 11:34 UTC (Thu) by dlang (guest, #313) [Link] (1 responses)

by the way, a note on the number of users and the "most machines don't need this feature argument"

If you count by the number of people at a keyboard of a machine you may be right.

but when you start considering server farms where the admin to system ratio is commonly well over 100 servers per admin, you would get very different numbers.

embedded systems are in many ways far more like servers than desktops. They have skilled admins managing the software and all the non-trivial configurations on the devices (either at the factory, or through 'firmware updates'

'winning the desktop' while loosing servers and embedded devices would not be a overall win.

Well, there are big difference...

Posted Nov 24, 2011 14:23 UTC (Thu) by khim (subscriber, #9252) [Link]

but when you start considering server farms where the admin to system ratio is commonly well over 100 servers per admin, you would get very different numbers.

Sure. For example you can get... zero. We have totally different systems on desktop and on servers here (they only share few common packages like glibc and libstdc++ to simplify testing). It does not matter at all what the servers are using because desktops are using totally different Linux. I'm pretty sure we are not alone.

embedded systems are in many ways far more like servers than desktops. They have skilled admins managing the software and all the non-trivial configurations on the devices (either at the factory, or through 'firmware updates'

Again: there are many different types of embedded systems and they use different packages. What they usually don't use are desktop-based distributions.

'winning the desktop' while loosing servers and embedded devices would not be a overall win.

Why not? Apple ceased to produce their rack-mountable servers and they use quite different set of programs on their iPad and Macbook - and still is fine AFAICS.

Sure, we should not forget servers, routers and other Linux-based machinery, but I fail to see why the desktop-oriented distribution should be driven by the needs of server or an embedded system.

I think this one thing which Ubuntu did right (and Fedora followed): distribution's work is selection first and foremost. It must work fine in the default install. If you want to serve a different niche - you can create a spin-off. Journald does not propose to remove support of syslogd for the time being and I'm pretty sure such support will only be considered in the future if Journald will be good enough for most uses, not just for desktop.

As for "who's the most numerous"... it's silly question: RedHat's accounting department knows the numbers, I'm pretty sure needs of the most important customers will be taken into account when the fate of journald will be determined.

Change... to what?

Posted Nov 21, 2011 22:58 UTC (Mon) by HelloWorld (guest, #56129) [Link]

> So why aren't you trying to change that?
Well, Lennart seems to be doing a good job at creating a better alternative to current systems, so why bother?

> GNU has the goal of remaking a Free UNIX.
That doesn't mean everything in Unix is perfect and must never be changed. In fact, GNU has made several design decisions that are different from SysV or BSD, for example they chose a microkernel design for the Hurd.

> Seems to me that if you disagree with pretty much all of the underlying design decisions of UNIX
I don't. I didn't comment on the idea of "everything is a file" at all, and I don't oppose the idea of a textual format in general. I'm just not making a religion out of it like you do. Now, I'm asking you again: what advantage would a text-based IPC/RPC protocol actually buy you, so that making it more inefficient than it needs to be is worth it?
And it's the same with the journal. I'm pretty sure it'll offer tools to dump logs in a textual format, so that you can use all the Unix tools you like with it. So, what's wrong with using a binary format when it has the potential to be more compact and much more efficient to query than a textual format?

> It is about time for a fork; to drive the heretics from the temple. To drive a stake in the ground and say here we take a stand; NO MORE. The Windows refugees who want 'a Windows that doesn't suck.' can go forth and do that. Elsewhere. If that means we never get 'the year of Linux on the Desktop.' we have to live with that consequence. We have made it twenty years waiting for that miracle and managed to thrive, if it never comes it isn't a terrible thing. Maybe UNIX just isn't for everyone, but for those who DO love it we would kinda like to keep it something we can recognize.
Dude, you have serious issues. Unix is a technology, it's not a religion. Get help.

Change... to what?

Posted Nov 21, 2011 23:05 UTC (Mon) by anselm (subscriber, #2796) [Link] (8 responses)

Who went away and put you in charge of deciding what is Linux and what isn't? There is no reason why we must stick forever with the arbitrary design decisions of the 1970s and 1980s. Lots of aspects of Linux are worth re-thinking. This does not mean that the first alternative proposal for »whatever« must be accepted by everyone, but also that there are no holy cows. (And that you are free to maintain Holy Cow Linux, with none of the newfangled improvements, for however long you care to do so.)

If somebody (not necessarily Lennart Poettering) comes up with a proposal to improve a notoriously badly-designed part of the system such as the logging infrastructure, then that person should at least be heard out. It may just be that they have something good going. That only Lennart and Kay seem to have the cojones to actually tackle some of these things should really give us pause.

In other words: Change is not necessarily for the better but it is also not necessarily for the worse. Biology has a word for things that never change. They're called »dead«.

Change... to what?

Posted Nov 23, 2011 23:16 UTC (Wed) by dlang (guest, #313) [Link] (7 responses)

when you say that syslog is so horribly designed and so limited it makes me think that you haven't used a modern syslog (rsyslog or syslog-ng). Those are both good examples of projects that have added a lot of capabilities to logging, but have done so in ways that are compatible with the existing system while providing the additional power to those who want/need it

this journal proposal sounds like the dozens of logging systems I've seen proposed by junior programmers for their applications. They assume that they know all the needs and so can design the perfect monolithic logging infrastructure.

to paraphrase a common quote

"syslog is the worst logging design ever, except for everything else"

Change... to what?

Posted Nov 25, 2011 4:33 UTC (Fri) by HelloWorld (guest, #56129) [Link] (6 responses)

> when you say that syslog is so horribly designed and so limited it makes me think that you haven't used a modern syslog (rsyslog or syslog-ng). Those are both good examples of projects that have added a lot of capabilities to logging, but have done so in ways that are compatible with the existing system while providing the additional power to those who want/need it
The Journal _is_ compatible with the syslog(3) api. And while it doesn't implement the syslog protocol, it is easily possible to write a gateway tool that implements the protocol and forwards messages to the Journal in order to be stored; in fact this possibility is mentioned in the proposal.
It would be really helpful if people would actually _read_ such proposals before whining about them.

Change... to what?

Posted Nov 25, 2011 5:34 UTC (Fri) by dlang (guest, #313) [Link] (5 responses)

I have read the proposal

It's "compatible" with syslog in that if you are willing to throw out it's advantages of being structured and trusted you can encapsulate syslog data in journal messages.

but if you have done this, what exactly is the advantage of this new tool?

and if you can go the other way and have the journal entries put into nicely formatted structured syslog messages, why do you need the journal to do this again instead of just writing the nicely formatted messages to syslog in the first place?

Especially if you like writing your log entries via a library, there is absolutely nothing preventing you from using a library that writes structured messages to syslog.

Change... to what?

Posted Nov 25, 2011 9:18 UTC (Fri) by anselm (subscriber, #2796) [Link] (3 responses)

Especially if you like writing your log entries via a library, there is absolutely nothing preventing you from using a library that writes structured messages to syslog.

Except that clients will still be able to lie to the syslog service about their identity, etc. Structured messages are one thing that The Journal is about, but Lennart and Kay do address a fair number of other concerns. Some of these might be amenable to changes to rsyslogd, but others are more difficult to treat within that framework while Lennart and Kay do seem to have a handle on them. I think their proposal at least bears thinking about.

Change... to what?

Posted Nov 25, 2011 9:25 UTC (Fri) by dlang (guest, #313) [Link] (2 responses)

>> Especially if you like writing your log entries via a library, there is absolutely nothing preventing you from using a library that writes structured messages to syslog.

> Except that clients will still be able to lie to the syslog service about their identity, etc. Structured messages are one thing that The Journal is about, but Lennart and Kay do address a fair number of other concerns. Some of these might be amenable to changes to rsyslogd, but others are more difficult to treat within that framework while Lennart and Kay do seem to have a handle on them. I think their proposal at least bears thinking about.

actually, journald is only able to avoid the application lieing to it for programs that are started by systemd and do all their logging through systemd.

This could be done with normal syslog by creating a unique /dev/log replacement for each application group that's started and then tagging to indicate which one the log arrives on (modern syslog daemons already allow you to have multiple sources and to know which source the log came from)

If anything else generates logs to journald in any way that bypasses the library, that thing could also lie about who it is. good programs won't do this, but good programs won't lie about who they are in the first place (unless they have a good reason to do so, which is very possible)

Change... to what?

Posted Nov 25, 2011 9:58 UTC (Fri) by michich (guest, #17902) [Link] (1 responses)

It does not matter what library you use to talk to journald. It is not possible to lie about who you are. Search for SCM_CREDENTIALS in the proposal. See also SCM_CGROUPS, SCM_COMM in http://0pointer.de/blog/projects/plumbers-wishlist.html

rsyslog now has 'trusted properties'

Posted Dec 1, 2011 3:43 UTC (Thu) by dlang (guest, #313) [Link]

it turns out that rsyslog already used SCM_CREDENTIALS to let you log the pid of the program, it's now added the ability to log some additional values

_UID The userid under which the logging process is being executed.
_GID The group id under which the logging process is being executed.
_PID The PID of the logging process. Note that this PID, if configured, is also put into the syslog tag.
_EXE Path to the binary that is logging
_COMM The name (as visible by top) of the logging process.
_EXE The full command line of the logging process. Note that this string can contain spaces, thus it is always provided in quoted form.

http://blog.gerhards.net/2011/11/trusted-properties-in-rs...

http://www.rsyslog.com/what-are-trusted-properties/

now that didn't take scrapping all existing logging practice to get did it?

Change... to what?

Posted Nov 25, 2011 9:20 UTC (Fri) by michich (guest, #17902) [Link]

Even with the syslog(3) api you still get some of the benefits: addition of the underscore-prefixed fields, indexing by these fields, per-user read access to their own logs.

Change... to what?

Posted Nov 21, 2011 4:35 UTC (Mon) by HelloWorld (guest, #56129) [Link] (1 responses)

Oh, by the way: you may want to check out something from ESR's book about the Unix philosophy:
http://catb.org/~esr/writings/taoup/html/ch01s06.html#id2...

"Rule of Diversity: Distrust all claims for 'one true way'."

I guess that applies to text formats and protocols as well.

Change... to what?

Posted Nov 26, 2011 16:39 UTC (Sat) by cdmiller (guest, #2813) [Link]

Indeed, it also applies to all the recent new and shiny proposals being touted as the 'one true way'...

Change... to what?

Posted Nov 21, 2011 6:23 UTC (Mon) by dvdeug (subscriber, #10998) [Link]

Looking sceptically at things because they're new is hardly as resistant to change as it gets. You're looking at things, you're open to change, you're just not going to jump on just anything. Considering that esd worked for me for almost ten years, and Pulseaudio isn't working nearly as well, I've got to say I like that idea.