| |||||||||||||
Missing the AF_BUSMissing the AF_BUSPosted Jul 5, 2012 4:39 UTC (Thu) by hp (subscriber, #5220)Parent article: Missing the AF_BUS
Do keep in mind that there are a number of avenues to improve D-Bus in userspace as well; I'm mystified why so much effort has gone into kernel changes and near zero into userspace:
http://lists.freedesktop.org/archives/dbus/2012-March/015...
but in any case the opportunity is there. My belief is that nobody is chasing this because for the vast majority of people D-Bus performance is not an actual problem. When I've asked for concrete examples of when it was a problem, things like Nokia N900 (iPhone 3G era hardware right?) come up, and poorly coded applications aren't ruled out and seem likely to be involved even in that case.
basically there is just no need to performance tune the kind of stuff dbus is normally used for on a stock Linux desktop... if something is only 1% of user-visible speed, making it double fast isn't perceptible.
people do show up on the mailing list using dbus on low resource embedded systems and needing ultra low latency or something, but in those cases dbus was pretty clearly a poor choice of hammer for the nail at hand.
I don't think Alan is wrong though. the dbus semantics and guarantees that make it slow are also what make it convenient, and app developers generally want those guarantees and apps are less buggy if they have them. So it might be nice to make this genre of thing fast, even if the simple notifications etc. used by the desktop aren't performance critical, there are other domains that might benefit from ordered, reliable delivery, lifecycle tracking, etc. there's no question a faster implementation of dbus would be more broadly useful beyond just the desktop.
(Log in to post comments)
Missing the AF_BUS Posted Jul 5, 2012 5:32 UTC (Thu) by hp (subscriber, #5220) [Link]
Also it's worth noting again that the design tradeoffs in dbus are almost all copied from X11. Messages between two processes are guaranteed to stay in the order sent, network errors are handled by just disconnecting and starting over, there's a central server that all the apps use to talk to each other, it's a binary protocol, there's a "selection" (bus name) concept used to acquire/release/locate resources, similar authentication mechanisms, server is located via environment variable, blocking round trips are discouraged (mechanisms are provided to avoid them), people use both almost exclusively on same-machine or at least a LAN, etc.
So most problems and solutions that apply to X11 will also apply to dbus.
I think the tradeoffs and guarantees made here are a pretty good guide to what desktop/mobile app developers want when they're writing a UI that's implemented as a "swarm of processes" (as all the X desktops are). Framed another way, this is what a local IPC system has to provide in order to support relatively reliable application code in this context. However, these tradeoffs are probably inappropriate for systems distributed over the internet or even over a cluster.
Based on dbus list traffic there seem to be development situations where similar tradeoffs make sense but the inherent slowdown of the central dispatch daemon is a problem. That's where kernel-accelerated dbus-like-thing would make sense maybe.
Missing the AF_BUS Posted Jul 5, 2012 9:17 UTC (Thu) by kyllikki (subscriber, #4370) [Link]
Concerning your comments on userspace D-Bus improvements. Collabora employ one of the upstream D-Bus maintainers and pay for some of their time to work on D-Bus.
We most definitely are committed to improving the userspace side of D-Bus in addition to the kernel work (which was a project for the GENIVI alliance)
Our eventual aim using all the solutions is for a tripling in throughput and a significant reduction of latency for the general case.
Missing the AF_BUS Posted Jul 5, 2012 10:33 UTC (Thu) by smcv (subscriber, #53363) [Link]
> poorly coded applications aren't ruled out
On the system bus, which is a trust boundary, poorly- or even maliciously-coded applications can never be ruled out, unfortunately.
> in those cases dbus was pretty clearly a poor choice of hammer for the nail at hand
People consider D-Bus to be a suitable transport for all sorts of things, desktop or not. The first sentence of the specification describes it as "a system for low-latency, low-overhead, easy to use interprocess communication", which probably contributes to the view that it's the right hammer for every nail - in practice, its current design tradeoffs tend to prioritize "easy to use" over low-latency.
Improving its latency, and avoiding priority inversion between the dbus-daemon and its clients, certainly increases the number of situations where D-Bus can be used. They might not be necessary for "the desktop bus", but that's by no means the only thing people use D-Bus for.
Improving the kernel-level transport is orthogonal to improving the user-space part (of which message (de)serialization is indeed likely to be the lowest-hanging fruit), and there's no reason they can't both happen.
> the dbus semantics and guarantees that make it slow are also what make it convenient
I absolutely agree that the convenient semantics - multicast signals, total ordering, conventions for lifecycle tracking and so on - are what make D-Bus valuable, and if you're willing to sacrifice those convenient semantics for performance, that's a sign that D-Bus is not right for you. Having said that, given the constraints of those semantics, the more efficient the better, and AF_BUS proves that there is room for improvement.
Missing the AF_BUS Posted Jul 5, 2012 13:01 UTC (Thu) by hp (subscriber, #5220) [Link]
> poorly- or even maliciously-coded applications can never be ruled out,
> unfortunately
What I meant here was, an app with lots of round trips in its protocol design or that shovels loads of data over the bus is going to be a perf problem. As a practical matter if you have user-visible situation xyz that appears slow, fixing dorky app behavior can be the fastest way to fix xyz.
> there's no reason they can't both happen
That's why I keep lobbying for the userspace changes to happen - a couple of them looked like they'd only take a few days of work. Hey, for all I know, someone did them already over the last few months. Anyway it's just bugging me (as you've no doubt gathered) that the kernel stuff is a kind of multi-year undertaking due to the difficult political issues, while the performance could be greatly improved without blocking on kernel devs...
So I'm just trying to give the potential userspace tasks some PR. Maybe someone reading these comments will want to work on them, we can dream...
(I know I'm telling those close to dbus things they already know. But it may not be apparent to those who aren't close to it that there's stuff they could do today.)
Missing the AF_BUS Posted Jul 5, 2012 17:15 UTC (Thu) by smurf (subscriber, #17840) [Link]
Apparently people now try to do some things across DBus that require a higher data rate and lower latency than the traditional "huh, somebody plugged in a USB stick / wants root for aptitude/yum / left fpr lunch" events DBus handled in the past.
Newfangled stuff like multi-touch. Or keyboard input (for Chinese and whatnot).
You don't want that stuff to go through more context switches (and processes) than strictly necessary. So AF_BUS seems to be a Good Thing.
Missing the AF_BUS Posted Jul 5, 2012 19:26 UTC (Thu) by iabervon (subscriber, #722) [Link]
It seems to me like the right solution for a lot of the low-latency or high-throughput applications is actually to send a socketpair endpoint over DBus, rather than using DBus for the actual data. Provided, of course, that that works. If nothing else, it would be very much worthwhile to save the packet framing when you have a logical data stream, and it's hard to beat a scheme where your Chinese user input can be turned into stdin with nothing more than dup2 and inherited by child processes.
Missing the AF_BUS Posted Jul 5, 2012 19:53 UTC (Thu) by hp (subscriber, #5220) [Link]
Yep. You can even use the dbus protocol over that socketpair. libdbus could probably provide some nice convenience API to facilitate this where you'd basically get a DBusConnection back from a method call to the remote service.
The downside is mostly that it's a fair bit more work for apps to do stuff like this. Services don't necessarily need to track "registered clients" right now but with this kind of setup they have to, in addition to dealing with the raw sockets and other extra work.
A lot of the discussion of speeding up dbus is motivated by trying to make the easy thing work well for apps, instead of requiring app authors to sort out these tradeoffs.
Especially with the higher-level API in say glib's dbus support, though, it might be possible to near-automatically put certain objects on dedicated sockets. Just a matter of programming...
Missing the AF_BUS Posted Jul 6, 2012 5:51 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]
It would be nice to be also able to use DBUS over the network. If you stick to DBUS-only protocol then it's not a problem.
But the minute you start using socketpairs - it becomes impossible.
Missing the AF_BUS Posted Jul 6, 2012 6:24 UTC (Fri) by smurf (subscriber, #17840) [Link]
socketpair-ing the sender and recipient would probably work, but it has a few downsides.
* you lose the easy debugging and monitoring you now get with DBus (presumably, with AF_BUS you could use something like wireshark),
* the client now has to juggle multiple file descriptors, that requires an API change
* multiple file descriptors and reliable message ordering don't mix
Too many downsides if you ask me.
Missing the AF_BUS Posted Jul 6, 2012 6:58 UTC (Fri) by Fowl (subscriber, #65667) [Link]
Surely libdbus could handle this all automatically?
Missing the AF_BUS Posted Jul 6, 2012 16:18 UTC (Fri) by smurf (subscriber, #17840) [Link]
The multiple-file-descriptors problem is the killer,
dbus_connection_get_unix_fd() returns exactly one file descriptor. If you open a direct connection to some application, you have more than one file descriptor. How do you make your application select()/poll() on both (or more) of these?
How would you propose to handle the monitoring problem? Let the daemon send a "somebody's listening for messages X, so if you exchange any of those privately, kindly send me a copy" commands to each and every client? Owch.
I'm not saying this cannot be done. I'm saying it's heaps more work, and more fragile, than simply moving the main chunk of this into the kernel, especially since there's already code which does that. And code is more authoritative than English.
Missing the AF_BUS Posted Jul 6, 2012 17:32 UTC (Fri) by iabervon (subscriber, #722) [Link]
I think I was unclear: I'm not suggesting that DBus do some sort of socketpair thing to provide high-speed dbus links; I'm suggesting that use cases that don't really care about DBus, but need to negotiate with another program for their input, could get back to their normal operation (input from a fd) by having the other side send a fd back when it's all set up. That is, user input doesn't normally go over DBus, so applications are already designed around a non-DBus input path; even if you need IPC to set up your input device appropriately, it doesn't make too much sense to get user input over DBus, particularly if you can't get corresponding user input over DBus if the system doesn't have Chinese input or multi-touch.
Of course, it's certainly possible that people will want high-speed IPC with DBus properties also, and it makes sense for DBus to be efficient regardless of whether it's running into performance constraints. But it doesn't make sense to use DBus for all communication, even if its performance could be made good enough.
Missing the AF_BUS Posted Jul 26, 2012 22:41 UTC (Thu) by oak (subscriber, #2786) [Link]
> When I've asked for concrete examples of when it was a problem, things like Nokia N900 (iPhone 3G era hardware right?) come up
One of the worst issues is D-BUS message delivery reliability. All it needs is an app that subscribes for some frequent message (like device orientation) and then doesn't read its messages either because it was supended or just buggy. As message delivery needs to be reliable, D-BUS will then just buffer the messages and get all the time slower and slower as it starts to swap.
Second issue is too complicated D-BUS setup. I think e.g. the N900 call handling goes through half a dozen daemons before the call UI pops up. Each of these steps adds it's own socket buffering and process scheduling overhead in addition to other overheads (e.g. paging the processes in to RAM if they were swapped out etc).
Then there's the D-BUS daemon code itself. Ever wondered why something that's "just" supposed to read and write data from sockets is CPU bound instead of IO bound? D-BUS daemon spends a lot of CPU on message content marshaling.
Missing the AF_BUS Posted Jul 26, 2012 22:57 UTC (Thu) by hp (subscriber, #5220) [Link]
the tradeoffs in the first issue can be configured in the config
file, it can throw errors when the buffer size is whatever you like. there are also some list/bug discussions of other behaviors that could be useful to support.
the second issue is not dbus's fault. that kind of thing is often from making a daemon when a library would be better. it's a bug in the app design.
the third issue I've mentioned repeatedly myself including in the threads I linked before.
but none of these three thing are concrete examples of user visible operations. in most real world cases all three of these problems are gotten away with and it isn't perceptible. n900 is the most often mentioned case where they aren't and if you're correct here, N900 has at least one really bad setup with half a dozen daemons.
|
|||||||||||||