Kernel prepatch 4.1-rc1

Posted Apr 29, 2015 20:04 UTC (Wed) by mbt (guest, #81044)
In reply to: Kernel prepatch 4.1-rc1 by johannbg
Parent article: Kernel prepatch 4.1-rc1

What is the huge rush? Why would corporate contributers like to see kdbus so soon?

As has been pointed out in several comments on this page, there are severe performance problems when using D-Bus as a mechanism to transfer big blobs of data. Also, as well pointed out in other comments, the speed improvement when using kdbus is less than an order of magnitude greater than that when using D-Bus. There are inefficiencies in the code (which could be improved without changing the ABI) and, evidently some arise (see http://lwn.net/Articles/642212/) as a result of having to support certain features. Changing such features would indeed entail an ABI change. That in itself is worrying: Linux makes ABI-stability guarantees, so ripping out a badly done component is a no-go. Also, should there be an improved D-Bus interface with the changed feature set, there would most certainly have to be code in the kernel to translate between old and new kdbus flavors. Can you think of a major commercial OS with this kind of problem?

Much has been made of the ten-year history of D-Bus and of the two years of kdbus development. Do you mean to tell me that over all this time no one has done the needed performance tuning that is magically supposed to happen because the code is in the kernel?

This brings us to what I suspect is the Real Reason for all the rush. The comment about the two kernel-contributing companies that want to see a quick merging of kdbus is a telling one. Vendors are forbidden from linking non-free programs to GPL-licensed libraries. They might try to get around this by using the message bus to communicate. Ah, but drat! D-Bus is also GPL licensed. So how to take advantage of GPL'ed programs and libraries but by the magic of syscalls. Kdbus is the ticket: the can use it to talk to GPL'ed code without violating the license. They can even use GLibC bindings: GLibC is LGPL.

It is handy thing that so many companies are able to take advantage of Linux and contribute to it. We gain from that, but when people want to play proprietary games, we all lose.

Kernel prepatch 4.1-rc1

Posted Apr 29, 2015 20:12 UTC (Wed) by pizza (subscriber, #46) [Link] (5 responses)

> Vendors are forbidden from linking non-free programs to GPL-licensed libraries. They might try to get around this by using the message bus to communicate. Ah, but drat! D-Bus is also GPL licensed.

While DBus-the-daemon is GPL'd, that is not true of many (most?) of the client libraries.

So, please, let's keep this discussion fact-based?

Kernel prepatch 4.1-rc1

Posted Apr 29, 2015 22:21 UTC (Wed) by mbt (guest, #81044) [Link] (4 responses)

I am indeed looking to the known facts while trying to uncover what appears to be a hidden agenda. So if D-Bus itself already offers a way to avoid GPL violations, my suggested explanation goes away. That leaves me casting around for other explanations. The performance improvement is the biggest reason people cheer for kdbus, but as Linus and several others have pointed out, the argument is specious. If kdbus gets into the kernel and is still too slow, then what?

Might it be better to use the message bus only for control? The cooperating processes could do handshaking over the message bus in order to set up an IPC mechanism much more performant than the message bus could ever possibly be. (There well **always** be overhead in communicating through the message bus; it's unavoidable!) You might as well stick with D-Bus.

From the D-Bus FAQ (http://dbus.freedesktop.org/doc/dbus-faq.html): The overhead comes from a couple of places; part of it is simply "abstraction penalty" (there are layers of code to support multiple main loops, multiple transport types, security, etc.). Probably the largest part comes from data validation (because the reference implementation does not trust incoming data). It would be simple to add a "no validation" mode, but probably not a good idea all things considered.

There are concerns with ABI stability, interactions with kernel components, and security. I'm left wondering why kdbus is so important. Please give us an answer that is better than "just because." I suspect that the true answer is one that the kdbus proponents would like not to admit.

Kernel prepatch 4.1-rc1

Posted Apr 29, 2015 22:57 UTC (Wed) by cesarb (subscriber, #6266) [Link] (3 responses)

> I am indeed looking to the known facts while trying to uncover what appears to be a hidden agenda. So if D-Bus itself already offers a way to avoid GPL violations, my suggested explanation goes away. That leaves me casting around for other explanations. [...] I'm left wondering why kdbus is so important.

I recalled the true reason for kdbus *not* being performance, but I couldn't recall precisely what it was (something related to containers IIRC), so I went back to one of its initial announcements, and here's what I found (https://lwn.net/Articles/580194/):

> Beyond that, credential passing is limited, there are no timestamps on messages, D-Bus is not available at early boot, connections to security frameworks (e.g. SELinux) must happen in user space, and there are race conditions around the activation of services.

This old presentation I found (https://archive.fosdem.org/2014/schedule/event/anatomy_of...) goes into more detail.

Kernel prepatch 4.1-rc1

Posted Apr 29, 2015 23:47 UTC (Wed) by dlang (guest, #313) [Link] (2 responses)

kdbus doesn't have any connections to security frameworks (dbus does at this point, so this is actually a regression)

As far as race conditions go, examples please? the one provided on the discussion says that services must not exit if there are any outstanding messages to be delivered to them and that's racy today. This seems like the wrong way to do things (want to prevent a service from updating to a new, more secure config, keep sending it messages)

why can dbus not start from initramfs? we've moved so many functions out of the kernel into initramfs, why is dbus more significant than RAID configuration?

Kernel prepatch 4.1-rc1

Posted Apr 30, 2015 1:10 UTC (Thu) by cesarb (subscriber, #6266) [Link] (1 responses)

> As far as race conditions go, examples please?

From what I understood, if I am recalling it correctly, the race condition was with inetd-style activation of dbus services. Something like:

1. The monitor process receives a message for the service, and spawns the service daemon.
2. The monitor process hands over the bus endpoint to the service, so any further messages go to the service daemon.
3. The service has been idle for a while, and decides to exit.
4. The monitor process notices that the service has exited, and starts monitoring the endpoint again.

Between 3 and 4, any messages sent to the service get lost.

Kernel prepatch 4.1-rc1

Posted Apr 30, 2015 1:32 UTC (Thu) by dlang (guest, #313) [Link]

if there is guaranteed delivery, why would the routing process not detect
that the message it sent to the service didn't get acknoledged?

what happens if the process dies instead of processing the message?

how does the kernel close this race? the kernel is not going to be starting
services and doing the other work that you describe the monitor process
doing.

Kernel prepatch 4.1-rc1

Posted Apr 29, 2015 20:28 UTC (Wed) by dlang (guest, #313) [Link] (4 responses)

Since RedHat doesn't sell desktop systems, why would they care about a _desktop_ buss?

Kernel prepatch 4.1-rc1

Posted Apr 29, 2015 20:58 UTC (Wed) by d9 (guest, #102181) [Link] (2 responses)

They use it in init and now want to add it to initramfs, apparently they don't like the scripts they currently have there.

http://www.spinics.net/lists/kernel/msg1977814.html

Kernel prepatch 4.1-rc1

Posted Apr 30, 2015 13:32 UTC (Thu) by johannbg (guest, #65743) [Link] (1 responses)

Better to view it as an reasonable request from an reasonable initramfs maintainer for many years and provide a link to his explanation of it [1] then to list this as some requirement by Red Hat

1. http://www.spinics.net/lists/kernel/msg1977695.html

Kernel prepatch 4.1-rc1

Posted Apr 30, 2015 22:30 UTC (Thu) by artem (subscriber, #51262) [Link]

Even better to view it via this link: http://thread.gmane.org/gmane.linux.kernel/1930358/focus=...

which shows several responses to that request. If you are really interested, the thread also continues here:

http://thread.gmane.org/gmane.linux.kernel/1940497

Kernel prepatch 4.1-rc1

Posted Apr 30, 2015 13:47 UTC (Thu) by HelloWorld (guest, #56129) [Link]

D-Bus isn't and never was a Desktop (only) bus.