Weekly Edition
Return to the Development page
| |||||||||||||
Williams: Eat Burgers on the Short Bus
Dan Williams, developer of NetworkManager, looks at D-Bus on his blog. In particular, he has found himself periodically giving mini-lessons about D-Bus; he captures that in this explanation of D-Bus concepts along with a concrete example of the interprocess communication (IPC) mechanism.
"service: a program that responds to requests from clients. Each service is identified by a 'bus name' which clients use to find the service and send requests to it. The bus name usually looks like org.foobar.Baz. A program can claim more than one bus name; NM claims org.freedesktop.NetworkManager and org.freedesktop.NetworkManagerSystemSettings, each is a unique service which provides different functionality to clients."
(Log in to post comments)
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 17:29 UTC (Mon) by arjan (subscriber, #36785) [Link]
another thing that many people forget about DBUS is performance.
DBUS was never intended for "hot path" communication or for really really large messages.
If you're designing something that depends on doing a constant (as opposed to init time + rare-ish changes) stream of dbus communication you need to really consider ways to mitigate this.
As a rule of thumb (and yes this is somewhat, but not very, pessimistic, but it makes you think about it the right way): a round trip over dbus takes 3 milliseconds. And you don't want to send more than a kilobyte or two over the bus per message; it gets encoded into a bus format, then decoded/encoded and decoded. And you want the encoded form (which is larger) to fit into a single 4Kb page to keep performance reasonable.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 18:23 UTC (Mon) by flewellyn (subscriber, #5047) [Link]
I thought DBUS was meant to be a generic, universal IPC mechanism? If it's got such poor performance, does this not mean it's failing at its job?
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 18:36 UTC (Mon) by dcbw (guest, #50562) [Link]
It doesn't have "such poor performance" as such. It's a great mechanism for IPC. It's not a great mechanism for pushing megabytes of data through quickly, simply because it's not meant for that and most programs don't do that sort of thing through IPC. There's a clear tradeoff between security and throughput in D-Bus, and D-Bus comes down on the side of security at the expense of being able to push MB of data through a connection quickly. Which most stuff simply doesn't do.
D-Bus certainly isn't failing at its job, it's actually great at its job. But its job does not involve high-throughput communications which traditionally IPC hasn't been much used for anyway, especially on consumer systems.
There are mechanisims in D-Bus like file handle passing that are designed to neatly address this problem.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 19:22 UTC (Mon) by flewellyn (subscriber, #5047) [Link]
Ahh, well, that's fair enough.
So for IPC which requires high throughput, a Unix domain socket or mmaped file would be a better choice?
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 19:27 UTC (Mon) by johill (subscriber, #25196) [Link]
I once started designing a shm-based transport for dbus for pushing large amounts of data like image scans (because SANE has got to have the worst API ever). In fact only the data itself has to be put into shm, it's not really a hard problem to solve.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 19:46 UTC (Mon) by atai (subscriber, #10977) [Link]
What about the disk format used by DConf, which supposedly is usable for IPC as well?
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 22:49 UTC (Mon) by arjan (subscriber, #36785) [Link]
IPC is expensive, no matter what format you use.
It means context switches, talking to different cpu cores, getting them out of idle/deep power save where needed, security checks, often format thunking (think 32/64 bit combined systems), format validation, a fair amount of cache thrashing etc etc.... to the central dispatcher daemon which then needs to make a routing decision. And then from that daemon to the destination. And then the reply messages goes through all of that again in reverse order (security checks, waking cpus up again, refilling the cache etc ;-)
That's not criticism of dbus or its design or format, but just the reality of having a powerful IPC layer.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 22:53 UTC (Mon) by atai (subscriber, #10977) [Link]
What about the IPC mechanism used under Android (Binder or such) which overlaps with DBus on purpose. It should have the same issues or being kernel based it is more efficient?
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 6:07 UTC (Tue) by linusw (subscriber, #40300) [Link]
In a thread at LKML I remember Marcel Holtmann mentioned that there were pending patches for D-Bus to pass file descriptors (could be a pipe and hence an kernel data passing buffer, shmem), that would help out in the case for passing large data. However it won't affect latency as far as I can see.
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 9:35 UTC (Tue) by wingo (subscriber, #26929) [Link]
D-Bus can pass file descriptors.
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 7:18 UTC (Tue) by ikm (subscriber, #493) [Link]
> It means context switches, talking to different cpu cores, getting them out of idle/deep power save where needed, security checks, often format thunking (think 32/64 bit combined systems), format validation, a fair amount of cache thrashing etc etc....
It still doesn't look nowhere near 3 milliseconds. Heck, a ping to my ISP over ADSL link takes 7 milliseconds!
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 9:47 UTC (Tue) by nye (guest, #51576) [Link]
>Heck, a ping to my ISP over ADSL link takes 7 milliseconds!
Wow. A ping to my ISP over ADSL takes 35ms and I thought that was good.
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 10:30 UTC (Tue) by ikm (subscriber, #493) [Link]
Yes, it's a 20mbit ADSL2+ link. Point is, it shouldn't be that DBUS local latency could be compared to a complex broadband networking scheme (computer->adsl modem->several km of wires->dslam->server and back) and found to be only twice faster. That just doesn't make sense.
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 11:53 UTC (Tue) by nye (guest, #51576) [Link]
Yes, sorry, I just wanted to register my jealousy.
I imagine it is the encoding into 'a bus format' and back again which takes the majority of the time, rather than the factors you quoted in your earlier post. I read Arjan's post as pointing out all the extra overheads that people often ignore, and which add up when you look at them all together, rather than a list of 'this are all the things that take the time' (and 3ms was only given as a worst-case example).
A ping over an ADSL connection, even allowing for traversal through a stack of network protocols, is still not a lot of work; it just happens to be sent over a long wire. I wonder what percentage of the ping time is in processing versus the time taken for the signal to physically propagate...
However, is anyone really trying to use dbus to sent a substantial flow of data? I've always thought of it as a way to send occasional messages like 'a USB storage device has been inserted; please mount it', or 'a global hotkey was pressed; please pause the music now'.
DSL latency Posted May 11, 2010 12:17 UTC (Tue) by farnz (guest, #17727) [Link] DSL uses a signalling rate of 4 kilobaud (and a stonking number of bits per symbol to get your megabits per second data rate). I can't remember off the top of my head if the superframe structure effectively requires you to see multiple symbols before you can pass data up to higher layers, but if it doesn't, this gives DSL a theoretical minimum ping time of 0.5 milliseconds (0.25 milliseconds in each direction), assuming that there is no processing delay. If you're really, really interested, the most of the ITU standards for DSL are available for download; G.992 is the ADSL family. Be warned that working out just how many symbols you need to get a ping through isn't simple - it's not laid out that clearly.
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 12:59 UTC (Tue) by ikm (subscriber, #493) [Link]
You know, the funniest thing here is that it's the dbus specification itself which states that dbus is supposed to be low-latency and low-overhead, and can be used for "potentially high-resolution same-machine IPC" after all:
> * D-Bus is low-latency because it is designed to avoid round trips and allow asynchronous operation, much like the X protocol.
> * D-Bus is low-overhead because it uses a binary protocol, and does not have to convert to and from a text format such as XML. Because D-Bus is intended for potentially high-resolution same-machine IPC, not primarily for Internet IPC, this is an interesting optimization.
http://dbus.freedesktop.org/doc/dbus-specification.html
Yet it was found to be 18 times slower than CORBA (http://eleceng.dit.ie/frank/rpc/CORBAGnomeDBUSPerformance...).
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 21:30 UTC (Tue) by njs (guest, #40338) [Link]
As always, "low" and "high" depend on your frame of reference. On latency and encoding overhead, it certainly beats out something like XML-RPC.
It isn't hard to be faster, though -- for instance, you just make direct connections between processes instead of going through a broker (which is how CORBA usually works, and probably explains why it's lower latency in that test you quoted). But then you lose out on many of D-Bus's advantages (security, simplicity of API/implementation/deployment, etc.).
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 21:36 UTC (Tue) by ikm (subscriber, #493) [Link]
> certainly beats out something like XML-RPC.
Totally. Anyway, dbus is here to stay. Whatever it is what it is. And it looks better than CORBA by any standard anyway (yuck!).
Williams: Eat Burgers on the Short Bus Posted May 12, 2010 12:44 UTC (Wed) by nix (subscriber, #2304) [Link]
Except that CORBA IDL is quite pretty compared to the XML horror of DBus.
DBUS secure? Posted May 10, 2010 23:53 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
I thought that DBUS suffered from the problem that there was no way to validate that someone sending a message through DBUS is who they claim to be?
DBUS secure? Posted May 11, 2010 5:22 UTC (Tue) by dcbw (guest, #50562) [Link]
Incorrect; there are multiple ways to validate the request. First, you can get the UID of the process making the request, because the D-Bus daemon uses Unix socket credentials to get it. If socket credentials are flawed, then a whole lot of stuff (not just D-Bus) is in trouble. The UID can then be used to authenticate the request.
Second, D-Bus can use SELinux context to validate the request; so even if the UID is compromised, the caller can still be restricted by SELinux context and thus limit any potential damage.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 19:58 UTC (Mon) by arjan (subscriber, #36785) [Link]
It's not so much bad at the "IPC" job as that it's not designed (nor should it be) to be free.
Large messages cost you. High frequency messages cost you.
That btw is true of pretty much all IPC that is not specifically targeting either of those two aspects. (And those that do have significant restrictions and usually pretty hairy programming models).
Basically "generic IPC == expensive for high frequency or large messages", and DBUS is no exception to this.
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 11:45 UTC (Tue) by nix (subscriber, #2304) [Link]
Note that to anyone who struggled with the hell that was CORBA, DBus is both simple and pleasant to use (it doesn't dictate a horrible object model bearing no resemblance to that in any useful language, for instance) and blazingly fast.
The only thing I'd really like is a proxying service so I could cluster several machines together and have them share dbus messages on the session bus (for KDE4, of course). This seems like it should be a SMOP (with a bit of extra complexity to handle fd passing): has anyone done it yet? (yes, there would be security concerns if this was done crudely: simply having the proxy run as an ssh subsystem would fix *that*.)
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 17:55 UTC (Mon) by maks (subscriber, #32426) [Link]
why can't you restart that monster?
now that hal is gone, can't dbus not also get nuked.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 18:41 UTC (Mon) by dcbw (guest, #50562) [Link]
You mean "why can't you restart D-Bus"? You actually can; the services need to handle a bus restart though, and any clients that depend on those services need to handle the bus restart and then re-acquire any cached state they had before the bus restarted. Though in practice if the bus is gone for a second or two, your cached state is probably still valid and you can just minimally verify it. But if the bus is gone for a long time, a network client would for example have to re-query the network service for the list of network interfaces (since a card could have been added/removed during the D-Bus restart), get the state of each of those interfaces, get the IP configuration of each of those, etc. It can get complicated. There are also ways the dbus daemon could handle a bus restart like pausing the connections and unblocking them after the core daemon has resumed.
I'm also not sure why you'd want to nuke D-Bus? It provides a very essential function that, if not present, would have to be re-written every single time somebody makes a daemon that provides a service. And the functionality that it provides (type-safety, security validation, multi-point messages, etc) is not easy to get right.
Is there some concrete issue you have where D-Bus does not meet your needs?
Williams: Eat Burgers on the Short Bus Posted May 11, 2010 7:16 UTC (Tue) by TRS-80 (subscriber, #1804) [Link] "the services need to handle a bus restart though" which would be easier if the default D-Bus library didn't exit(2) when the bus disappears.
Williams: Eat Burgers on the Short Bus Posted May 12, 2010 9:43 UTC (Wed) by k8to (subscriber, #15413) [Link]
Reliability is for windows.
Williams: Eat Burgers on the Short Bus Posted May 13, 2010 11:12 UTC (Thu) by HelloWorld (guest, #56129) [Link]
Well, you can use atexit to register a function that does a longjmp. It is a horrible workaround for a bad library though.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 20:37 UTC (Mon) by michaeljt (subscriber, #39183) [Link]
Many people seem to use DBus for things it isn't really suitable for, as Arjan said in his comment above. hal seems to me to be a good example - using a userspace daemon to keep an in-memory database of installed hardware which is queried via a desktop IPC server was not what I would call an elegant design, especially as a good number of the things hal did could have been accomplished by a simple library (and I don't know how many of the rest even needed DBus). I'm not even convinced by the use of DBus for starting applications like PolicyKit does - I think in most cases a setuid tool to do the starting would be just as good (that is, I think that most of the security in PolicyKit comes from its authors' hard work, not from the fact that it uses DBus).
As a way of sending notifications, kick messages or whatever to running daemons, applets and things, I think DBus is great.
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 19:01 UTC (Mon) by Trelane (subscriber, #56877) [Link]
dcbw: since you're about, thanks for the great writeup! I particularly like the pseudocode at the end. However, I'm curious about how you'd go about implementing the *server* you're talking to. Could you please post a follow-up on this subject?
Thanks again!
Williams: Eat Burgers on the Short Bus Posted May 10, 2010 20:28 UTC (Mon) by cmccabe (guest, #60281) [Link]
Great writeup Dan!
This might be a dumb question, but has anyone written a FUSE filesystem that shows you all the object paths in D-BUS?
Python D-Bus Example Posted May 11, 2010 5:28 UTC (Tue) by ldo (subscriber, #40946) [Link] I did some example Python code using D-Bus at CodeCodex.Com.
|
|||||||||||||