LWN: Comments on "The D-Bus Broker project"

The D-Bus Broker project

mathstuf — Tue, 29 Aug 2017 21:53:58 +0000

Well, if it were me, I'd make sure the state could be decorated with a `#[derive(Deserialize, Serialize)]` and then just shove it over a pipe and initialize it that way ;) . File descriptors can follow along afterwards. It's a pity C doesn't have something as easy (maybe CapnProto or its ilk?).

The D-Bus Broker project

jiiksteri — Tue, 29 Aug 2017 16:48:36 +0000

> The current situation, where an upgrade of the dbus daemon (e.g., due to a security issue) requires a reboot, is a serious pain point.

Given that it's possible to pass fds across a unix domain socket with SCM_RIGHTS messages, wouldn't a seamless upgrade be possible with some co-operation between the old and new versions? In theory at least.

Of course all that requires you to quiesce the broker enough so that its state can be enumerated and serialized. And you're stuck with maintaining the fragile upgrade protocol, which is soon more code than the actual broker. And you'll find yourself needing to consider the upgrade protocol every time you change some internal state details in the broker.

Suddenly you have a whole new IPC mechanism in your IPC mechanism and everything is miserable :)

The D-Bus Broker project

bpearlmutter — Tue, 29 Aug 2017 09:43:37 +0000

This is really a killer feature, perhaps *the* killer feature. The current situation, where an upgrade of the dbus daemon (e.g., due to a security issue) requires a reboot, is a serious pain point. Especially when systems are set up to do unattended upgrades for security-critical packages.

The D-Bus Broker project

kreijack — Mon, 28 Aug 2017 18:08:22 +0000

DBus Broker would be able to support a restart ? I.e. serializing its state on disk, restart the executable and load the old state ? If I try to restart DBus, everything break.
I know that it is difficult due to the connection, however it is useful in case of an update.

The D-Bus Broker project

flussence — Sat, 26 Aug 2017 15:15:23 +0000

>Why is _new_ system software still developed in C? Shouldn't Rust or Go be the default choice for new projects at this point?
These programs are written by humans, you're not offering a magic wand that can go s/C/Rust/g on the decades of programming knowledge stored in their head, and quite frankly most people have started to tune out unsolicited suggestions like this as spam.

The D-Bus Broker project

flussence — Sat, 26 Aug 2017 15:01:51 +0000

>What world would have made "/org/kde/kspread/sheets/3/cells/4/5" useful to technical endusers as it's the example in the dbus introduction?
That looks a bit like XPath. Now I'm stuck imagining what the world could be like if we just had a systemwide DOM API instead of D-Bus...

The D-Bus Broker project

liam — Sat, 26 Aug 2017 02:46:36 +0000

I know this is bad form, but that was a great comment:)

The D-Bus Broker project

njs — Sat, 26 Aug 2017 00:51:25 +0000

There are lots of architectures where you can run Linux, but where Rust has missing or limited support. You may or may not care about these architectures in any given project, the Rust toolchain is getting better at this over time, etc., but even if you're only interested in Linux then it is still an issue to think about.

The D-Bus Broker project

ncm — Fri, 25 Aug 2017 20:43:00 +0000

A better question might be, "Shouldn't C++ be the default choice for new projects?"

Rust and Go are still too immature in any number of ways -- portable to too few target architectures, too uncertain of future vigor, too little-known, with too few independent implementations, and too little third-party tool support -- to be a wise choice of language for ambitious projects today. In ten or twenty years each might be mature enough (or fading, and weren't you wise to have avoided it?). While it should be obvious to all that C is too problem-ridden to be a good choice for new projects, and is frankly too expensive to code in, C++, used well, happily avoids all of C's problems, is as mature, portable, and vigorous as anyone can ask for, and is markedly *faster* than C, besides.

But perhaps an even better question is, why should any new project depend on D-bus, and confine itself to such a limited target base? The overwhelming majority of programming targets worldwide have some sort of Linux kernel, but no D-bus. While D-bus depends on Posix/Linux-specific features, nothing D-bus does couldn't be done just as well in some other messaging system that could work equally well in many environments.

Maybe a better choice for the future would be to adapt current D-bus subscribers to work with whatever such apparatus Android provides, and implement a Free version of that, with a published wire protocol to enable multiple language bindings.

The D-Bus Broker project

micka — Fri, 25 Aug 2017 08:47:17 +0000

No, really, I don't think HTML is a good language to implement low level OS plumbing.

The D-Bus Broker project

lsl — Fri, 25 Aug 2017 03:06:20 +0000

> For infrastructure pieces of software it's also imo not a good choice to use go, because you can't easily access libraries written in it from other languages.

This hasn't really been the case for quite some time now. You can export a C API from your Go code and provide it in the form of a static archive or shared library. The requirement that the Go runtime has to provide the executable's entry point is long gone.

Still, I wouldn't be surpised if putting this inside a single process together with, say, a JVM might uncover some interesting effects. OTOH, it seems to work fine on Android.

The D-Bus Broker project

clopez — Fri, 25 Aug 2017 01:30:42 +0000

This project claims to be Linux-only on purpose. Portability is not an argument in this case

The D-Bus Broker project

liam — Thu, 24 Aug 2017 22:11:31 +0000

3.2. Priority of Constituencies
In case of conflict, consider users over authors over implementors over specifiers over theoretical purity. In other words costs or difficulties to the user should be given more weight than costs to authors; which in turn should be given more weight than costs to implementors; which should be given more weight than costs to authors of the spec itself, which should be given more weight than those proposing changes for theoretical reasons alone. Of course, it is preferred to make things better for multiple constituencies at once.

HTML Design Principles

The D-Bus Broker project

dtlin — Thu, 24 Aug 2017 21:54:45 +0000

I think it meets its original ambitions. D-Bus frees up developers from developing their own ad-hoc IPC systems over and over again. The end effect is not that users see the underlying technology, but that they get more or better apps.

The D-Bus Broker project

andresfreund — Thu, 24 Aug 2017 21:53:48 +0000

> Why is _new_ system software still developed in C? Shouldn't Rust or Go be the default choice for new projects at this point?

At the moment the portability just isn't there for some types of projects. For projects intended to run on a wide variety of platforms, including ones which aren't widely popular but still commercially relevant in some fields (say AIX), you really can't use rust or go. For infrastructure pieces of software it's also imo not a good choice to use go, because you can't easily access libraries written in it from other languages. And rust still has build performance issues. There's project intentionally forgoing c++ because the support across platforms isn't great... (I think that's going too far though). I think go partially isn't there due to NIH syndrom (e.g. static linking, old virtual memory discussions). Rust seems more promising.

Probably more controversially, I think it's also currently not necessarily viable to write complex lower level pieces of software in either yet. I spend most of my time working on RDBMs and some of the low-levely stuff isn't quite there imo, although I have high hopes.

The D-Bus Broker project

tuna — Thu, 24 Aug 2017 20:39:30 +0000

"Why is _new_ system software still developed in C? Shouldn't Rust or Go be the default choice for new projects at this point?"

Why do people think that Rust and Go are similar and have the same use cases?

The D-Bus Broker project

pizza — Thu, 24 Aug 2017 20:33:39 +0000

Sure, I can always open a TCP socket to Google and manually construct the XML-RPC OATH requests to authenticate tokens against their systems, but why would I do that by hand when I can just use a tool already written that provides a higher-level API to hide all of those messy details?

(At some point it becomes necessary to care about those details -- in NetworkManager's case, using nmcli means there's no way to subscribe to events, for example)

The D-Bus Broker project

alogghe — Thu, 24 Aug 2017 20:12:45 +0000

That's a neutral observation.

If you think about it, you'll realize it has nothing to do with anything I was talking about.

Would you use dbus to automate a server deployment?

When looking at the ansible sources it appears the answer is "not really".

Ansible uses dbus+nmcli to list the interfaces and then any manipulation appears to be all shell calls.

I'm not anti-dbus, I just think its fallen far short of its original ambitions. Maybe that's fine with the people using it these days.

The D-Bus Broker project

smurf — Thu, 24 Aug 2017 20:07:48 +0000

Yeah, but the problem is that it's apparently far easier to fork off nmcli than to open a dbus socket to talk to NetworkManager, even though that would be a whole lot more efficient and whatnot.

The D-Bus Broker project

pizza — Thu, 24 Aug 2017 19:24:24 +0000

nmcli interacts with the NetworkManager daemon using the same set of d-bus APIs that any other application can use if they want to interact with NetworkManager. It's an excellent example of a project eating its own dogfood.

The D-Bus Broker project

Cyberax — Thu, 24 Aug 2017 19:23:25 +0000

nmcli uses dbus internally.

The D-Bus Broker project

alogghe — Thu, 24 Aug 2017 19:07:25 +0000

Actually NetworkManager is a great concrete example.

In 2017 most people would go directly to nmcli to automate any change to their network settings.

In 2005-2006 I think the dbus people hoped for different?

Maybe dbus has found the limits of its niche and, as I said, that's great.

The D-Bus Broker project

jcpunk — Thu, 24 Aug 2017 18:53:35 +0000

I realize I'm a bit obsessed with AMQP, but could it be a way forward (and an implicit way to add network aware behavior if so inclined) rather than building a new standard behavior set?

The D-Bus Broker project

richard77 — Thu, 24 Aug 2017 18:49:50 +0000

Thanks for the answer.
I misunderstood the paragraph regarding "direct activation".

The D-Bus Broker project

alogghe — Thu, 24 Aug 2017 18:46:30 +0000

Yes, I didn't want to type the paragraph you typed in the middle of my sentence.

The D-Bus Broker project

nybble41 — Thu, 24 Aug 2017 18:32:17 +0000

> It [D-Bus] seems to have settled in as a useful tool to people with commit rights on the kde and gnome git.

D-Bus is used for more than just KDE and Gnome. For example, udisks, systemd, NetworkManager, PolicyKit, and PulseAudio all expose APIs through D-Bus. While these services are used _by_ KDE and Gnome, they all have command-line interfaces (besides e.g. dbus-send or qdbus) and GUIs for other graphical desktop environments.

The D-Bus Broker project

alogghe — Thu, 24 Aug 2017 18:04:39 +0000

The "api" is the surface area of all the programs that expose a Dbus interface. The ecosystem and it's utility to it's users.

My question is overly broad and probably out of scope as I stated.

What world would have made "/org/kde/kspread/sheets/3/cells/4/5" useful to technical endusers as it's the example in the dbus introduction?

https://www.freedesktop.org/wiki/IntroductionToDBus/

When those docs were written people seem to have had broader ambitions.

I don't see people (mostly engineers) that use Linux desktops ever pointing to Dbus as a solution to their problems in 2017.

I don't see people writing server apps that speak it to each other.

I don't see end user applications making much of it.

It seems to have settled in as a useful tool to people with commit rights on the kde and gnome git. That's actually great if that's the limit of it's goals.

In using dbus in writing applications I've found libraries implementing it verbose, underutilized by applications (example- it was easier to just use an X11 binding to monitor window titles), and when I walk away from that section of the codebase it takes some work to reengage with "dbus" when I get back to it.

My response to it isn't "I should use more dbus", it's "I should rip this out and use grpc".

Even if I made a great desktop app that everyone used daily, I don't see that they would use a dbus interface that I put a bunch of work into making.

The D-Bus Broker project

luto — Thu, 24 Aug 2017 15:31:34 +0000

Semi-shameless plug: it might be worth benchmarking on tip/x86/mm. Context switches mostly don't flush the TLB on that branch, and this type of workload may be significantly faster if everything is on the same CPU.

The D-Bus Broker project

mathstuf — Thu, 24 Aug 2017 14:49:00 +0000

There's already a marshalling library: https://github.com/srwalter/dbus-serialize

The D-Bus Broker project

dvdhrm — Thu, 24 Aug 2017 14:17:39 +0000

> From the "Deviations" document linked in the blog post I understand that dbus-broker requires systemd. Is that correct?

No, that's not correct.

`dbus-broker` is the binary that implements the D-Bus Message Bus and it has no mandatory dependencies other than libc.

However, if you want the compatibility launcher, which provides integration compatibility to dbus-daemon, including parsing their desktop-file format, their xml-configuration, and supporting systemd-activation, then yes, you need glib, libsystemd, and libexpat.

IOW, we strictly split between the dbus-broker and the integration part. We optionally provide a default integration, so people can actually deploy the broker. Our main focus is the dbus-broker, though, and we're committed to support any other 3rd party integration.

The D-Bus Broker project

richard77 — Thu, 24 Aug 2017 12:36:56 +0000

From the "Deviations" document linked in the blog post I understand that dbus-broker requires systemd. Is that correct?

The D-Bus Broker project

joib — Thu, 24 Aug 2017 11:47:08 +0000

s/buy/by

Pointless sentence here to get past LWN 10-character comment limit.

The D-Bus Broker project

joib — Thu, 24 Aug 2017 11:42:30 +0000

No, just pointing out that as far as "uncomfortable experience" is a/the metric to go buy, the continued use of memory unsafe languages like C comes at a cost as well.

The D-Bus Broker project

jrigg — Thu, 24 Aug 2017 11:25:09 +0000

So you're offering your services to the project as a Rust programmer?

The D-Bus Broker project

joib — Thu, 24 Aug 2017 10:56:40 +0000

Unless you're a psychopath, your users getting p0wned due to yet another buffer overflow exploit is also an uncomfortable experience. ;)

The D-Bus Broker project

dvdhrm — Thu, 24 Aug 2017 10:30:29 +0000

> bus1 has some interesting properties that D-BUS misses, like guaranteed causality.

True. bus1 guarantees causality even if you use side-channels, as long as the cause and the final effect are both bus1 operations. D-Bus cannot guarantee this, as it has an intermediate hop.

But just to be clear: D-Bus guarantees total global order, which implies causal order, but only regarding D-Bus interactions. Side-channels (like two peers communicating via unix-signals, etc.) are not considered.

> It looks like it can also replace Android binder (?).

Theoretically, yes. Practically, who knows... Lots of politics involved there.

The D-Bus Broker project

dvdhrm — Thu, 24 Aug 2017 10:24:56 +0000

> The D-Bus Api is unpleasant to work with, what could be done to fix that?

What is "The D-Bus API"? D-Bus is a specification and there are many different libraries that provide the most different APIs. If one of them is unpleasant to work with, try a different one? If all of them are awkward, maybe you can suggest an API that would be nicer?

I bet all IPC library writers are more than eager to find the perfect API, but it is no easy job. And this problem is not limited to D-Bus.

> What is the overlap with the Android intents system? What could be learned or repurposed?

I don't see an overlap with the Android intents system. It is implemented on an upper layer. D-Bus provides the IPC and recommends use-cases. What you end up doing with it is beyond the scope of D-Bus.

> My sense is that Dbus mostly works for people that write KDE and Gnome but it isn't for many others.
> Applications writers should find it easy to use and expose their applications to dbus.
> Linux users should be able easily browse and consume those services such that application writers are encouraged to write good dbus services.

This is not really a question, but a mere statement. Yes, we can do better in exposing more convenient APIs, or better language integration. Help welcome!

D-Bus is in no way perfect. I think the blog post tries to describe some of its shortcomings (and we are aware of many more). But D-Bus is around and not going away, and I am not aware of a serious competitor, yet. A lot of us are working on better alternatives, but unless they gain traction, we intend to simultaneously improve D-Bus and make sure the worlds stay compatible.

The D-Bus Broker project

dvdhrm — Thu, 24 Aug 2017 10:06:13 +0000

I completely agree.

We actually considered writing the marshaling layers of dbus-broker in Rust, but it takes more than a few months to feel confident enough in a language to actually ship the code. There are definitely plans to use Rust in ongoing work, though.

The D-Bus Broker project

mpr22 — Thu, 24 Aug 2017 09:34:08 +0000

Internalizing Rust's distinctive language properties (ownership and lifetimes) to the point that you find them "intuitive" takes time and effort, and until you've done so, writing nontrivial software in Rust is an uncomfortable experience for anyone who's accustomed to programming in an unsafe language.

The D-Bus Broker project

kragil — Thu, 24 Aug 2017 06:50:15 +0000

This is probably a unpopular comment on LWN, but:

Why is _new_ system software still developed in C? Shouldn't Rust or Go be the default choice for new projects at this point?

(IMNSHO They may not be as fast if you know what you are doing, but they are easier and safer for sure and they could produce software that is easier to maintain and they make modern multicore systems more accessable + more.)