The kdbuswreck [LWN.net]

The kdbuswreck

Posted Apr 28, 2015 6:42 UTC (Tue) by zyga (subscriber, #81533)
In reply to: The kdbuswreck by luto
Parent article: The kdbuswreck

The one thing that I think you may be missing is that kdbus-based dbus doesn't do much at all in the userspace deamon. It is the current design that does put all of the overhead in the one userspace process. With the kernel based version half of the overhead is removed outright (A->server->B->server->A becomes A->B->A in the common case).

Secondly, AFAIR, the current dbus daemon gets penalized by fair kernel scheduling. That issue goes away with kdbus. Lastly I think that it's prety clear that kdbus unlocks a whole new level of performance with code based on memfd that current dbus doesn't use.

Still, the threads you've referenced are interesting and I need to read more into them to understand how kdbus-based changes applies to them.

to post comments

The kdbuswreck

Posted Apr 28, 2015 7:07 UTC (Tue) by luto (subscriber, #39314) [Link]

I realize that a dbus-like design (central daemon relaying messages) will probably take a performance hit due to context switches and copies. However, there's no reason that a synchronous method call should need 15 context switches, nor is there any reason that dbus couldn't use memfd for large messages.

Regardless, this particular dbus benchmark is so incredibly slow that none of this explains it, and kdbus is apparently only twice as fast. I'm not sure what the problem is, but it's not the scheduler or the fact that there's a central daemon.

IOW, yes, kdbus is in principle twice as fast as a dbus-like design. But dbus is several hundred times slower than it should be. Let's fix that first before quibbling over the other factor of two by moving some or all of it into the kernel.

The kdbuswreck

Posted Apr 28, 2015 14:15 UTC (Tue) by granquet (guest, #60931) [Link]

>Secondly, AFAIR, the current dbus daemon gets penalized by fair kernel scheduling. That issue goes away with kdbus. Lastly I think that it's prety clear that kdbus unlocks a whole new level of performance with code based on memfd that current dbus doesn't use.

yes, I concur here.
The switch to the CFS broke some use cases at the place I was working at that time.

but probably, those use cases where a bit stupid ;)

The kdbuswreck

Posted Apr 30, 2015 16:01 UTC (Thu) by ksandstr (guest, #60862) [Link] (1 responses)

>Secondly, AFAIR, the current dbus daemon gets penalized by fair kernel scheduling. That issue goes away with kdbus.

The issue should've gone away when priority inheritance was mooted for AF_UNIX to support lower latency in Xorg: the scheduler should've been altered to also select the previous process' IPC peer ("partner") to run until the client's wakeup condition was satisfied, and then return to the client immediately. This would've made a closed wait over AF_UNIX equivalent to a syscall, some thousands of clock cycles notwithstanding.

It's my opinion that a transitive form of partner scheduling and priority inheritance would've made an userspace DBus daemon near-transparent from a performance point-of-view, were a sufficient "counts as partner call" boundary possible to distinguish from the many states and forms of I/O sleep found in Unix. However today, instead of a relatively simple and well-defined primitive behaviour (and perhaps a tiny control API to manage it), we have 10_000 lines of lennartware being pushed for inclusion -- and not in staging like Android's "binder", either.

And before someone else in our little peanut gallery chimes in about priority inheritance: while that is necessary for a well-performing IPC architecture, it's insufficient a solution to the whole of the latency issue because rather than re-using the abstract scheduling decision that made a client process run in the first place, it only elevates the recipient's priority. A scheduler may well schedule an unrelated process in the server's (elevated) priority band, for example. The inheritance mechanism's interactions with scheduling quantums (the server's? the client's? at what priority? for how long?) and its teardown conditions have also remained poorly defined, which suggests that these issues just cold-up aren't being considered.

Finally, to not call "lennartware" without justification, and based on the considerations above, it's my prediction that if kdbus is merged, there'll be a span of two to six years immediately afterward at the end of which that which remains of kdbus-2015 will not be a net loss to its applications anymore, as with PulseAudio and Avahi before that.

The kdbuswreck

Posted May 4, 2015 7:58 UTC (Mon) by dgm (subscriber, #49227) [Link]

> the scheduler should've been altered to also select the previous process' IPC peer ("partner") to run until the client's wakeup condition was satisfied, and then return to the client immediately. This would've made a closed wait over AF_UNIX equivalent to a syscall, some thousands of clock cycles notwithstanding.

Hear! Hear!

This has the potential to make requesting services from a daemon (any daemon) much more efficient. Everything from web servers to desktop environments could benefit. Just think about how many daemons are constantly running in any typical desktop (answer: dozens!)

One has to wonder why something like this doesn't exists yet?