The unveiling of kdbus
The good and bad of D-Bus
Unlike most other kernels, Linux has never had a well-designed IPC mechanism. Windows and Mac OS have this feature; even Android, based on Linux, has one in the form of the "binder" subsystem. Linux, instead, has only had the primitives — sockets, FIFOs, and shared memory — but those have never been knitted together into a reasonable application-level API. Kdbus is an attempt to do that knitting and create something that is at least as good as the mechanisms found on other systems.
Linux does have D-Bus, which he said, is a powerful IPC system; it is the
closest thing to a standard in this area as can be found on Linux. Lennart
put up an extensive list of advantages to using D-Bus. It provides a nice
method-call transaction mechanism (allowing for sending a message and
getting a response) and a means for sending "signals" (notifications) to the
rest of the system. There is a discovery mechanism to see what else is
running on the bus and the introspection facilities needed to learn about
![[Lennart
Poettering]](https://static.lwn.net/images/conf/2014/lca/LennartPoettering-sm.jpg)
what services are offered. D-Bus includes a mechanism for the enforcement of
security policies, a way of starting services when they are first used,
type-safe marshaling of data structures, and passing of credentials and file
descriptors over the bus. There are bindings for a wide range of languages
and network transparency as well.
On the other hand, D-Bus also suffers from a number of limitations. It is well suited to control tasks, but less so for anything that has to carry significant amounts of data. So, for example, D-Bus works well to tell a sound server to change the volume, but one would not want to try to send the actual audio data over the bus. The problem here is the fundamental inefficiencies of the user-space D-Bus implementation; a call-return message requires ten message copies, four message validations, and four context switches — not the way to get good performance. Beyond that, credential passing is limited, there are no timestamps on messages, D-Bus is not available at early boot, connections to security frameworks (e.g. SELinux) must happen in user space, and there are race conditions around the activation of services. D-Bus also suffers from what Lennart described as a "baroque code base" and heavy use of XML.
Even so, Lennart said, D-Bus is "fantastic" and it solves a number of real problems. Ten years of use have shown that the core design is sound. It is also well established and widely used. So the right thing to do is not to replace D-Bus, but to come up with a better implementation.
Into the kernel
That implementation is kdbus, an in-kernel implementation of D-Bus. This implementation is able to carry large amounts of data; it can be reasonably used for gigabyte-sized message streams. It can perform zero-copy message passing, but even in the worst case, a message and its response are passed with no more than two copy operations, two validations, and two context switches. Full credential information (user ID, process ID, SELinux label, control group information, capabilities, and much more) is passed with each message, and all messages carry timestamps. Kdbus is always available to the system (no need to wait for the D-Bus daemon to be started), Linux security modules can hook into it directly, various race conditions have been fixed, and the API has simplified.
Kdbus is implemented as a character device in the kernel; processes wishing to join the bus open the device, then call mmap() to map a message-passing area into their address space. Messages are assembled in this area then handed to the kernel for transport; it is a simple matter for the kernel to copy the message from one process's mapped area to another process's area. Messages can include timeouts ("method call windows") by which a reply must be received. There is a name registry that is quite similar to the traditional D-Bus registry.
The "memfd" mechanism enables zero-copy message passing in kdbus. A memfd is simply a region of memory with a file descriptor attached to it; it operates similarly to a memory-mapped temporary file, "but also very differently." A memfd can be "sealed," after which the owning process can no longer change its contents. A process wishing to send a message will build it in the memfd area, seal it, then pass it to kdbus for transport. Depending on the size of the message, the relevant pages may just be mapped into the receiving process's address space, avoiding a copy of the data. But the break-even point is larger than one might expect; Lennart said that it works better to simply copy anything that is less than about 512KB. Below that size, the memory-mapping overhead exceeds the savings from not copying the data.
Memfds can be reused at will. A process that needs to repeatedly play the same sound can seal the sample data into a memfd once and send it to the audio server whenever it needs to be played. All told, Lennart said, memfds work a lot like the Android "ashmem" subsystem.
The signal broadcasting mechanism has been rewritten to use Bloom filters to select recipients. A Bloom filter uses a hash to allow the quick elimination of many candidate recipients, making the broadcast mechanism relatively efficient.
There is a user-space proxy server that can be used by older code that has not been rewritten to use the new API, so everything should just work on a kdbus-enabled system with no code changes required.
When will this code make its appearance? It has been announced on the D-Bus mailing list, and the code is available in the relevant repositories now. The main thing that is missing at the moment is the policy enforcement mechanism. Everything will work, Lennart said, if one doesn't mind that it will all be "horribly insecure." The plan is to get the code merged into the mainline kernel sometime in 2014. He is optimistic that this will work out; having Greg Kroah-Hartman involved in the process helps with his confidence there. But Lennart noted that two previous attempts to get D-Bus functionality into the kernel have failed, so there are no guarantees. Stay tuned over the course of the next year to see how it goes.
See kdbus.txt in the kernel-side source repository for more information on the design of kdbus.
[Your editor would like to thank linux.conf.au for funding his travel to
Perth].
| Index entries for this article | |
|---|---|
| Kernel | kdbus |
| Kernel | Message passing |
| Conference | linux.conf.au/2014 |
Posted Jan 13, 2014 21:50 UTC (Mon)
by cyperpunks (subscriber, #39406)
[Link] (10 responses)
[1]: https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&...
Posted Jan 13, 2014 22:04 UTC (Mon)
by rahulsundaram (subscriber, #21946)
[Link] (1 responses)
Posted Jan 13, 2014 22:10 UTC (Mon)
by alexl (subscriber, #19068)
[Link]
Posted Jan 13, 2014 22:32 UTC (Mon)
by ovitters (guest, #27950)
[Link] (2 responses)
Or in other words: you're wrong.
Posted Jan 14, 2014 21:05 UTC (Tue)
by marcH (subscriber, #57642)
[Link] (1 responses)
You are assuming you understood something.
Posted Jan 15, 2014 0:22 UTC (Wed)
by ovitters (guest, #27950)
[Link]
Posted Jan 14, 2014 2:56 UTC (Tue)
by mezcalero (subscriber, #45103)
[Link] (3 responses)
https://bugzilla.kernel.org/buglist.cgi?bug_status=NEW&...
Unfortunately I cannot tell you how many those are, because my browser choked on the immense number of them...
;-)
Posted Jan 17, 2014 1:13 UTC (Fri)
by deepfire (guest, #26138)
[Link] (2 responses)
I'm asking because 237 bugs is a lot for a Core OS, well, core component.
Yes, I realise that systemd is still under heavy development, but it's also extremely widely deployed and these bugs probably bite a lot of people -- so that I don't feel you can just shrug off the implications..
Posted Jan 17, 2014 4:52 UTC (Fri)
by rahulsundaram (subscriber, #21946)
[Link]
2) it is not 237 bugs as I have already pointed out earlier. Many of them are requests for enhancements, several unconfirmed issues, several cloned bugs, several tracker bugs etc.
Posted Jan 17, 2014 4:56 UTC (Fri)
by mathstuf (subscriber, #69389)
[Link]
I see 16 in the QA pipeline and 5 "post", so presumably these are fixed, just not in Fedora (stable) yet (down to 169). Maybe a third have substantial discussion (10+ comments), so I'll assume they're on their way.
That's still a lot and I'm just going to skim it for bugs that probably[3] don't belong (not many; maybe 10%): scan-build errors (#1047009; no report offered, just asking devs to use it since they found a lot), some probable RFEs (#1054473, #1054348), bash-completion-related bugs (#1024379), things that look harmless for actual usage (#1047148, #1008188), one pulseaudio-related (#928513), tmpfs-on-/tmp (#1028155), and some others. I also see duplicate bugs for journal-related problems (slowness when the logs get large, missing logs for some services) and packaging errors (#907719).
I'd guess there are maybe ~75 (reported) bugs with the systemd *package*, a handful of which are serious without much discussion about them (not sure if it's a bunch of "me too" or real discussion though). Does that sound better?
[1]https://admin.fedoraproject.org/pkgdb/acls/bugs/systemd
Posted Jan 14, 2014 10:48 UTC (Tue)
by johannbg (guest, #65743)
[Link]
We have had on average 50 bugs filed against systemd per Fedora release cycle.
With the trend of reports being one third RFE's, one third has been misfiled and one third been actual bugs.
As per maintenance policy those bugs *always* get fixed upstream then Michal or Zbyszek decide if and then to which GA releases those fixes will be backported to.
I'm hoping we get to the point where we can adapt rebase policy align with the kernel as well as adapting the bug handling Justin has been working on in the kernel sub-community which seems to be working afaikt.
If and when we get there you should see bugs in the number of <10 against systemd if any et all.
Posted Jan 13, 2014 22:00 UTC (Mon)
by tau (subscriber, #79651)
[Link] (35 responses)
Are there any plans to break this functionality out so that it can exist independently of kdbus for things like Wayland or mainlined Android? AFAIK the current state of the art here is to securely create a temporary filename in /dev/shm and then unlink it after opening (or do some hack with procfs to anonymously create a fd charged to the /dev/shm filesystem), then mmap that and pass the fd. However, tmpfs files don't have any facility like memfd sealing. It seems that this primitive may have applications in a number of other areas.
Posted Jan 13, 2014 22:22 UTC (Mon)
by wahern (subscriber, #37304)
[Link] (4 responses)
The problem with vmsplice, and presumably memfd, is that you can't easily reuse that memory. Which means you can only use it for a fixed number of objects throughout the process lifetime.
It would be nice if a lot of the new code is composable and can be used to accelerate things not related to D-Bus. If not, then it's a real shame, and a sign that Linux is being taken down the path of Windows and OS X, in terms of short-sighted, niche, "fad" APIs.
Posted Jan 14, 2014 3:05 UTC (Tue)
by mezcalero (subscriber, #45103)
[Link] (2 responses)
vmsplice() splices pages to a fifo, but there isn't a way to get the pages out of it without copying them. The stuff is also note reusable. You cannot mmap a fifo... and so on and so on.
An unlinked file in tmpfs is certainly much closer to the memfd concept than vmsplice() is. However the sealing concept it lacks, which is the major limiting factor making it unusable as a transport for D-Bus without requiring copying on the receiving side.
Lennart
Posted Jan 15, 2014 17:41 UTC (Wed)
by TD-Linux (guest, #92557)
[Link]
I'm currently writing a driver that basically exposes regions of memory to userspace, and the ioctl() controls mapping/unmapping (just like sealing memfds).
Because both memfds and streamable DMA buffers basically appear the same from userspace, maybe a standard API could be used to support both?
Posted Jan 16, 2014 15:53 UTC (Thu)
by bcrl (guest, #5934)
[Link]
Posted Jan 14, 2014 3:48 UTC (Tue)
by vrfy (guest, #13362)
[Link]
Posted Jan 13, 2014 22:25 UTC (Mon)
by alogghe (subscriber, #6661)
[Link] (29 responses)
Part of the point of (k)dbus is for applications to not reinvent the wheel with all the the other bits and pieces they will inevitably need.
Posted Jan 13, 2014 22:50 UTC (Mon)
by wahern (subscriber, #37304)
[Link] (28 responses)
Once upon a time Linus would frown on these sorts of single-use interfaces. He tended to prefer focusing on making other primitives more performant so that you could compose more complex interfaces in userspace. And for the kernel that's an excellent rule of thumb.
Remember the HTTP server wars? Microsoft had a blazing fast HTTP server in-kernel. So somebody coded one up in the Linux kernel, to one up Microsoft. But then somebody with infinitely more sense tweaked some kernel APIs and wrote an HTTP server in user-space that bested both the in-kernel servers.
Posted Jan 13, 2014 23:12 UTC (Mon)
by alogghe (subscriber, #6661)
[Link] (21 responses)
The reasoning for having some parts of dbus in the kernel are quite sound (good presentation from Lennart).
Posted Jan 14, 2014 0:35 UTC (Tue)
by wahern (subscriber, #37304)
[Link] (20 responses)
If AF_BUS was rejected, what was changed to make something like this more amendable for inclusion?
If there are "no blocking calls", what happens when you hit a kernel memory limit? Or is it simply the case that all I/O operations behave as-if O_NONBLOCK is set.
I understand that, fundamentally, you need kernel cooperation for performant (low-latency, low-power, high-throughput, etc) broadcast messaging. You don't necessarily need a complex naming schema in kernel, but that's kind of a grey area.
Posted Jan 14, 2014 0:54 UTC (Tue)
by corbet (editor, #1)
[Link] (18 responses)
Posted Jan 14, 2014 1:19 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link] (17 responses)
Posted Jan 14, 2014 8:56 UTC (Tue)
by edeloget (subscriber, #88392)
[Link] (16 responses)
Having something the size of kdbus with some kind of fixed guidelines on how to use it is IMHO a bad idea. There are situations where you want to devise your own protocol (i.e. while I always thought kdbus was a good idea, I also always thought that it would be socket-based and lightweight).
A multicast AF_UNIX socket makes more sense (possibly with something like memfd, which sounds quite interesting and would deserve a user space API on its own). Once you have both, implementing dbus in userspace is far less tricky and the resulting code should be both faster and cleaner.
Putting the dbus daemon and its complexity into the kernel is IMHO a bad idea (seriously: this kdbus moduledoes too many things).
Posted Jan 14, 2014 10:23 UTC (Tue)
by smurf (subscriber, #17840)
[Link] (13 responses)
Please either tell us how to achieve that with a multicast AF_UNIX and some (OK … a lot) of libdbus_mcast-ish scaffolding around it, or kindly shut up.
The people who did the kdbus work think that a userspace solution either won't work or would be significantly more complex than a kernel driver.
Posted Jan 14, 2014 20:22 UTC (Tue)
by edeloget (subscriber, #88392)
[Link]
Posted Jan 14, 2014 21:52 UTC (Tue)
by edeloget (subscriber, #88392)
[Link] (11 responses)
You ask me to tell you how I would implement some kind of dbus with AF_UNIX multicast alone. It seems to me that kdbus goes around that limitation by allowing programs to instantiate memfd objects. So an AF_UNIX multicast would probably use memfd too (and I already said that this idea sounds quite interesting).
Now, using the multicast mechanism to throw short messages (signaling) and memfd to pass data blocks make things quite sexy in the end. The libdbus (which is still needed and still have big things to do) can do the marshalling in and out - no need for a daemon to do that. Having the marshalling done in user space instead of kernel space does indeed add complexity to the library but it also removes this complexity from the kernel (not to mention that with your argument about complexity in the kernel to avoid complexity in user space, you are paving the way to the "glorious" day when you'll see that big emacs.ko land in staging/).
On that marshalling point: as some already said (on the mailing list, I believe), there are some "weird" field inversion and changes in the kdbus protocol when compared to the current dbus protocol. These changes will require some kind of marshalling in the library itself - so it's definitely possible.
Now that I've something with (a) zero copy (or a limited number of copies for the signaling info that foes thru the AF_UNIX multicast socket), (b) secure (because access to both the multicast socket and the memfd object can be kernel controlled), (c) looks like dbus (because, well, it's libdbus), (d) doesn't require a daemon process, can I explain what I believe is the biggest flaw in the current kdbus approach ? Or should I still shut up for eons and eons and not speak in front of your Glorious Person again?
When I say that the current implementation of kdbus does too many things, I really mean it. The fact is that the kdbus protocol is quite large and not very flexible. If kdbus is integrated into the kernel, this procotol will be part of the kernel ABI which means that it'll need to be supported for years, maybe decades. So the first point is to make sure that the defined protocol is complete and not broken - because an incomplete/broken, public protocol in the kernel is far harder to fix than a private (WRT the kernel), incomplete/broken protocol in an application. If you have to add one field in one central structure of the defined protocol then you may need to implement a new version of the protocol (possibly all the protocol, not just the change you wanted to make ; I failed to see any version field in any of the protocol structs) ; meaning that you'll have to support two protocols instead of one. Add one other error and you'll get a new one. In 5 years, you'll have multiple protocol versions to support and maintain - unless you can assure me that the full protocol is OK and will never change.
A large protocol also means that handling this protocol and the interraction between the protocol messages will be complex. It seems you are willing to put this complexity in the kernel when I think that complexity should be put far away from the kernel. The kernel in itself is already more complex and difficult to apprehend than any other program I know. This very complexity is often cited as a barrier that limit the possible participation of newcomers. If you come with a killer algorithm for the kernel scheduler you'll have a hard time producing a patch that will allow you to test and distribute your idea. Adding complexity where it can be avoided is therefore a bad idea.
@judas_iscariote: I hope I answered to your questions as well (and obiously, kdbus is not for me ; the question is not about me using it, me liking it, or someone else using it or liking it ; it's about the integration of kdbus in the Linux kernel, the pros, the cons, the possible issues in the current approach and alternatives).
Posted Jan 14, 2014 22:30 UTC (Tue)
by peter-b (guest, #66996)
[Link] (3 responses)
Anyway, so Greg etc. gave up on that idea, hence the kdbus driver.
Posted Jan 14, 2014 22:58 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link] (2 responses)
Posted Jan 15, 2014 7:54 UTC (Wed)
by alexl (subscriber, #19068)
[Link] (2 responses)
As for the no daemon part, you seem to have a limited knowledge of dbus. The things you list are not really enough for replacing dbus, unless you want the kernel to e.g. read dbus service config files and spawn processes as described in them.
Posted Jan 15, 2014 16:29 UTC (Wed)
by edeloget (subscriber, #88392)
[Link] (1 responses)
Which is exactly what I proposed, minus the metadata protocol (which needs its own marshalling) because I firmly believe that this protocol is
> As for the no daemon part, you seem to have a limited knowledge of dbus. The things you list are not really enough for replacing dbus, unless you want the kernel to e.g. read dbus service config files and spawn processes as described in them.
I'm not sure kdbus is going to handle that as well, unless the kdbus developpers want the kernel to e.g. read dbus service config files and spawn processes as described in them.
My previous mail was about proposing something that would replace the proposed implementation of kdbus (a) without the unneeded complexity of the proposed kdbus (b) using simple, reusable kernel mechanisms.
My point is that (kdbus with its internal memfd) can be replaced by (libdbus+multicast AF_UNIX+memfd) - the result is very likely to be more elegant (both in and out the kernel), less intrusive (kernel-wise) and less limited (change to regular AF_INET multicast and bam, you've got distributed dbus on a local network ; not sure it's a good idea but who knows?).
If you need additional functionalities like spawning a process, you can still have a dbus daemon but instead of being a central point in the design its goal will be to listen to a particular set of messages (i.e. it's an endpoint).
(BTW, I may not know dbus as much as some others ; I don't know music like Mozart did yet that doesn't prevent me to hear and recognize false notes).
Posted Jan 15, 2014 16:32 UTC (Wed)
by edeloget (subscriber, #88392)
[Link]
> because I firmly believe that this protocol is
... too complex and too fragile to fit in the kernel.
Posted Jan 15, 2014 19:37 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link] (3 responses)
The semantics of sockets and kdbus are quite different actually, and single-copy stuff with sockets cannot really be done...
I am pretty sure our current design is simpler, shorter, faster and more likely to be acceptable upstream.
Lennart
Posted Jan 15, 2014 20:22 UTC (Wed)
by nix (subscriber, #2304)
[Link] (2 responses)
Not that I've tried to use it for that, or anything, so I could easily be wrong.
Posted Jan 15, 2014 23:27 UTC (Wed)
by wahern (subscriber, #37304)
[Link] (1 responses)
Also, what do you do if a single process hangs and holds up the queue in your tree of tee(2) calls? Actually, I have the same question regarding kdbus, but it's been left unanswered--how does the flow control work with kdbus?
Posted Jan 17, 2014 17:03 UTC (Fri)
by nix (subscriber, #2304)
[Link]
Posted Jan 14, 2014 20:53 UTC (Tue)
by judas_iscariote (guest, #47386)
[Link] (1 responses)
For this cases, kdbus is not what you are looking for.
> Putting the dbus daemon and its complexity into the kernel is IMHO a bad >idea (seriously: this kdbus moduledoes too many things).
why do you think "does too many things" is an argument.. ?
Posted Jan 15, 2014 5:11 UTC (Wed)
by CameronNemo (guest, #94700)
[Link]
Posted Jan 14, 2014 3:11 UTC (Tue)
by mezcalero (subscriber, #45103)
[Link]
The presentation slides are available here btw:
Posted Jan 14, 2014 20:16 UTC (Tue)
by brugolsky (guest, #28)
[Link] (4 responses)
While my bias is in favor of your broader point about providing generally-useful orthogonal services to userspace, I believe that your recollection is faulty, and unfairly maligns the work that went into the TUX webserver. In the wake of the Mindcraft Benchmarks, which showed IIS / NT outperforming Apache / Linux 2.2 at web serving, Ingo Molnar leveraged the wide-ranging scalability improvements in the Linux 2.4 kernel to create the TUX in-kernel HTTP accelerator, which crushed everything that existed at that time. Less than a year later, the X-15 userspace webserver appeared and matched or slightly exceeded TUX's performance -- proving that the Linux syscall interface was complete and performant enough to be negligible for the SpecWeb99 workload.
The initial Windows in-kernel accelerator, http.sys, appeared later, in IIS 6.0 / Windows 2003.
TUX was popular for serving static content (in CDNs and other high-volume applications) for some time, but was eventually superceded by event-driven userspace webserver designs like nginx.
Posted Jan 14, 2014 21:29 UTC (Tue)
by PaXTeam (guest, #24616)
[Link] (3 responses)
Posted Jan 15, 2014 11:16 UTC (Wed)
by hummassa (subscriber, #307)
[Link] (2 responses)
Posted Jan 15, 2014 12:33 UTC (Wed)
by PaXTeam (guest, #24616)
[Link] (1 responses)
Posted Jan 15, 2014 13:20 UTC (Wed)
by hummassa (subscriber, #307)
[Link]
> maybe go out into the real world every now and then and ask server operators how they like to clean up after getting owned. so yes, when performance comes at the cost of security (we're talking about *remote* kernel bugs, do you even comprehend the seriousness of that?)
This seems to be the non-sarcastic part of your post, so I'll seriously respond:
Been there, done that, got many white hairs in interminable tech-versus-accounting discussions on the topic of IIS (ugh) versus Apache versus Tux and what was cheaper, to let our webserver be cracked and then restore the backups on a new one (oh, the early nineties, no need for cdns and stuff) or to harden them. (Hint: accounting wins all discussions, [joke] those discussions are a good rehearsal for marriage [/joke]).
Tux won for the two years where the profit of serving ten times more clients in the same hardware was greater than the cost of having a spare server semi-ready, rotating them from time to time, etc.
But, for what is worth, you are right that it was a dreadful time, security-wise. But it was before script kiddies, and before 9/11 and DHS and the NSA. We did telnet our machines at the time, all the time.
Posted Jan 23, 2014 16:35 UTC (Thu)
by psusi (guest, #95157)
[Link]
Posted Jan 13, 2014 22:42 UTC (Mon)
by tau (subscriber, #79651)
[Link] (62 responses)
From a purely technological standpoint I personally would definitely like to see systemd and its associated stack succeed, not just for the improved process management, but because it standardises a lot of previously distro-specific functionality and sticks an actual API on top of it. But it's not going to happen if Mr Poettering keeps marching into the room, declaring himself to be in charge and asking the volunteers who cross his path what orders they want to receive first. Or if it does, then it will happen with far more hard feelings than it strictly needs to.
Posted Jan 13, 2014 22:58 UTC (Mon)
by rahulsundaram (subscriber, #21946)
[Link]
http://lists.freedesktop.org/archives/dbus/2013-December/...
It continues well into Jan for those who want to follow and make up their own minds.
Posted Jan 13, 2014 23:43 UTC (Mon)
by ovitters (guest, #27950)
[Link] (11 responses)
Think you're biased, especially the way you describe things. I'm guessing you're not used to different communication styles?
Posted Jan 14, 2014 2:05 UTC (Tue)
by hazmat (subscriber, #668)
[Link] (10 responses)
Posted Jan 14, 2014 13:57 UTC (Tue)
by ovitters (guest, #27950)
[Link] (9 responses)
You agree that after 6 messages it got extremely messy? I read 30+ and saw none of it. Only after loads and loads more messages I saw the Canonical discussion. According to the kdbus developers (Lennart, Greg, Kay) it should NOT be done. They've suggested people try and write patches to understand this. Yet you so easily say that it is miniscule. Suggest to write patches instead. Further, as suggested in the same thread, this bit can be implemented by a LSM / linux security module. Your argument seem to boil down to "everything should be in the kernel" / "please everyone". LSM would be in the kernel.
Posted Jan 14, 2014 15:02 UTC (Tue)
by hazmat (subscriber, #668)
[Link] (3 responses)
reading through more of the messages with fresh eyes, its pretty clear that the kernel impl wants to be lean and clean with tight focus, which is perfectly normal. Also normal is the desire for a message bus to be a policy attachment point as its pretty natural and indeed there is discussion there from lennart on direct extension of the acl mechanism with selinux tags (http://thread.gmane.org/gmane.comp.freedesktop.dbus/15499). The issue is that for those wanting fine grained security (down to method invocations) the use of kdbus is going to be problematic without extension of the headers. The issue with patches here is that they've explicitly already stated they'll be rejected, so what's the point of producing them?
its not so much about putting everything in the kernel as it about an implementation than satisifies a general set of use cases such as those that the existing dbus already does. its unfortunate the previous impl didn't make it past the net maintainers (binder/ashmem compatible). anyways given the kdbus impl and the desire for not wanting to extend its routing headers with additional message attributes its clear that those who want fine grain will either have to implement payload parse in particular lsms or more likely fallback to dbus daemon and not get the benefit of kdbus.
Posted Jan 14, 2014 16:57 UTC (Tue)
by gregkh (subscriber, #8)
[Link]
Also, it allows you to be able to persuade the original developers that such a change really isn't a big deal.
And, it also might just prove to yourself that asking for such a feature, really was a bad idea as the patches to implement it was horrible and caused major problems.
That's why the Linux kernel mailing list very rarely has "idea" discussions, real patches are what matters.
Posted Jan 14, 2014 22:14 UTC (Tue)
by ovitters (guest, #27950)
[Link] (1 responses)
Initially they were quite firm on the no. But seems more like "no, that'll be bad", followed by a later "but convince me with code if you can". That's pretty much kernel style way of discussing design. Maybe too used to the kernel mailing list though (even though I haven't read it for many years, I rely on LWN).
I really like a flat out "no" though. Much clearer.
Posted Jan 15, 2014 10:53 UTC (Wed)
by jonnor (guest, #76768)
[Link]
Posted Jan 25, 2014 15:24 UTC (Sat)
by makomk (guest, #51493)
[Link] (4 responses)
Posted Jan 25, 2014 16:11 UTC (Sat)
by kmacleod (guest, #88058)
[Link]
Lennart's design goal appears to be (from his comments to the ml and irc) that there is only one user bus (and one process manager per user) and that each login on any graphical display be merged into one compositor+shell, xinerama-style.
The GNOME guys are insistent on having only one logical session (where session in this context means the compositor+shell doesn't treat multiple displays as independently usable workplaces).
It seems to me that a different compositor+shell could easily support merging multiple logins into one user bus (and one process manager) but still treat some displays as independent workplaces by presenting a menu/topbar/hot-spots and attached devices on separate workspace displays, even to the point of allowing someone to simply throw a running app from one workplace to another.
So having one user bus and one process manager per user seems like a very reasonable low-level approach that allows the desktop environments to handle multiple workplaces however they want.
Posted Jan 26, 2014 20:46 UTC (Sun)
by ovitters (guest, #27950)
[Link]
> Frankly, the whole thing reminds me of the tactics that some of the more
Frankly, that kind of comments are inappropriate.
Posted Jan 26, 2014 23:22 UTC (Sun)
by smurf (subscriber, #17840)
[Link] (1 responses)
Everybody else should just get attached to their existing session. That already happens today, you get a strongly-worded "either you use your existing session or something might break" warning (I cannot remember which desktop environment does that, as I usually only use the login screen directly after boot-up).
What's the problem, beside "it has always worked that way and now Lennart says it's nonsense, therefore Lennart is full of it"?
Posted Jan 27, 2014 1:50 UTC (Mon)
by kmacleod (guest, #88058)
[Link]
Use cases I see: As I said in my comment earlier, one user bus and one process manager per user/host is good, I'll go further to say one "instance" of GNOME/other DE is good too but there should be (probably freedesktop.org-level) support for multiple session-like support for different desktops, seats, workplaces, apps, etc. This might take the form of namespace hierarchies in the D-Bus user bus or, more likely across different machines, some other way to provide "localilty" to apps that need to know. X11 was originally intended to fulfill this and did extremely well for years (and for many legacy cases still works great). Over time as apps got closer to the hardware X started performing poorly and really doesn't work well for this anymore. Wayland and remote display protocols are much better placed for doing this today.
Posted Jan 14, 2014 1:43 UTC (Tue)
by smurf (subscriber, #17840)
[Link]
Posted Jan 14, 2014 3:25 UTC (Tue)
by mezcalero (subscriber, #45103)
[Link] (47 responses)
"Another project's mailing list out of nowhere" --- well, I have been the maintainer of the dbus package for a while before Simon took over. Not for long, and I didn't do a good job at it, except for doing a few releases, but well, I have contributed my small share to it.
Quite the opposite to that "first person" you are refering to as "the list", who contributed the staggering amount of ... zero patches to dbus over all those years....
That said, this game is stupid. Regardless whether somebody has contributed something in the past or is an "outsider", I am quite sure that in Open Source we should accept and review everybody's patches. At least for systemd we try hard to judge by quality of code and the concepts behind it, and not by anything else...
The discussion on the dbus mailing list was messy and certainly not overly productive (except for the good stuff Simon posted), however, try to see the from our perspective for an instant, if the first mails you get are from some random folks who never contributed anything, contain no technical comments but tell you that you shouldn't call this "dbus", and tell you that your code should not have been written in the first place.
Lennart
Posted Jan 14, 2014 10:19 UTC (Tue)
by tsmithe (guest, #57598)
[Link] (46 responses)
Yet -- watching these sagas evolve is fascinating (and by the number of comments they attract here on LWN, clearly I am not alone in thinking this). There is something Machiavellian in it: I don't understand all the technicalities (though I enjoy learning), and so I watch the power struggle.
It is a power struggle that seems to exist only because the people on the edges have willed it. I think you just want to write good software -- but because you have originated so much good software, your actions are misconstrued. And yet, now, because of such misperceptions, fights really do break out, and there really do seem to be clans of people pro and versus you.
It must be unsettling.
Posted Jan 15, 2014 14:12 UTC (Wed)
by NAR (subscriber, #1313)
[Link] (38 responses)
Posted Jan 15, 2014 15:20 UTC (Wed)
by johannbg (guest, #65743)
[Link] (6 responses)
If the alsa maintainers had manned up, stepped up and admitted this when people where flaming Lennart his reputation might be better intact.
The fact is Lennart has been working hard to make people lives easier but people are working hard to make his life harder by discouraging his work and the project he works ( which he never works on alone btw ) and that has gone far enough already and people should stop flaming Lennart period
Posted Jan 16, 2014 1:23 UTC (Thu)
by eean (subscriber, #50420)
[Link] (5 responses)
Which was why it was inappropriate for the distributions to deploy pulseaudio when it was deployed. There are other ways to fix ALSA drivers. You don't have the make sound unfun for thousands of users. :)
That said I don't see the same mistakes being made with systemd at all. It was completely unnoticed when I upgraded to the version where openSUSE started using systemd. I'm guessing it's a combination of lessons-learned and no dependency on device drivers.
Mostly the comparisons to PulseAudio is ad hominem attack on Pottering. I've never heard any substantive comparison. There's no equivalent to "broken ALSA drivers, but let's ship anyways" in systemd.
Posted Jan 16, 2014 3:01 UTC (Thu)
by mchapman (subscriber, #66589)
[Link] (3 responses)
I'm not sure if most of the problems would have been found if PulseAudio had had a more limited initial release. Sometimes you need thousands of users with thousands of different hardware combinations.
I'm wondering how many problems were actually ALSA bugs and how many were just due to the settings Ubuntu had chosen. The PA developers made it quite clear that those settings were suboptimal.
Posted Jan 16, 2014 14:41 UTC (Thu)
by mathstuf (subscriber, #69389)
[Link]
Posted Jan 19, 2014 2:22 UTC (Sun)
by krakensden (subscriber, #72039)
[Link] (1 responses)
I've heard this a million times but I've never actually seen a list of what they did wrong. I've honestly started to chalk it off more to PA-developer deflection than truth.
That said, I've certainly seen Debian and Ubuntu developers screw up packaging before.
Posted Jan 19, 2014 3:04 UTC (Sun)
by mchapman (subscriber, #66589)
[Link]
I don't use Ubuntu, so I only got to watch the debacle from the sidelines. It's rather hard to piece together the story now some 5 years later.
As far as I can tell it was a confluence of several problems.
* https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug...
* http://ubuntuforums.org/showthread.php?t=789578
There might have been other issues too. It's complicated, because this all coincided with the time PA implement so-called "glitch-free" audio, which as I understand it exercised a lot of code in the ALSA drivers that was rarely used before.
Posted Jan 16, 2014 15:08 UTC (Thu)
by HelloWorld (guest, #56129)
[Link]
Posted Jan 15, 2014 15:26 UTC (Wed)
by raven667 (subscriber, #5198)
[Link] (29 responses)
There is one lesson to be learned though, Lennart has a visceral rejection of cluttering up his services with workarounds when the problems are in other open-source software which can be changed to fix the bugs instead. His pushback to fix the bugs where they lie at the source has made the whole stack more robust.
What is unfortunate is that there is not a large enough desktop Linux ecosystem to have the vendors of consumer hardware do their own testing and bug fixes rather than requiring things to break on end-user desktops before bugs can be discovered and fixed.
Posted Jan 15, 2014 16:06 UTC (Wed)
by NAR (subscriber, #1313)
[Link] (4 responses)
Also, "no workarounds" is understandable from the engineer's perspective, but from the user's perspective it looks like "that *** broke my system and doesn't even want to fix it!". In proprietary environment this is the time when a manager intervenes one way or other to avoid public relation disaster.
Posted Jan 15, 2014 16:56 UTC (Wed)
by niner (subscriber, #26151)
[Link] (3 responses)
As an openSUSE user I found the whole discussions about Pulseaudio rather strange. It was just one easily discoverable click to turn Pulseaudio off when it didn't work and one click to turn it on again when the problems were fixed. And nowadays I have a wonderful system and can use the very nice features Pulseaudio provides.
Posted Jan 17, 2014 1:56 UTC (Fri)
by deepfire (guest, #26138)
[Link] (2 responses)
- no rollout strategy discussions were possible
Sorry, this view doesn't fly. Lennart had a meaningful power to specify the pulseaudio use scenarios -- but he didn't exercise it. Probably because everything worked fine on his systems.
I'm not saying it's an uncommon mistake -- but it's an uncommon situation as a whole -- the impact was tremendous, since, well, everybody uses sound.
Posted Jan 17, 2014 2:20 UTC (Fri)
by rahulsundaram (subscriber, #21946)
[Link]
On the contrary, he predicted that any such rollout will have to deal with issues and those issues would likely only surface with mass deployments of the software under different real world configurations due the wildly different set of hardware out there. This is the nature of complex software (compatibility layering requirements) and the amount of investment commercial Linux vendors are willing to make to move the desktop oriented technology forward.
Posted Jan 17, 2014 3:55 UTC (Fri)
by mathstuf (subscriber, #69389)
[Link]
Posted Jan 15, 2014 17:16 UTC (Wed)
by paulj (subscriber, #341)
[Link] (14 responses)
I fired up an old CPC emulator the other day, to play an old game I like. It used to play sound last time, via /dev/dsp. It doesn't today. Pulseaudio keeps /dev/dsp open, preventing others using it, even when not playing anything. The pulseaudio wrapper binary causes the whole thing to segfault somewhere.
Re the "Those weren't my bugs" attitude of earlier pulseaudio days, that might have been technically correct at some deep engineering level, but it wasn't at all what the end-user considered correct behaviour.
Posted Jan 15, 2014 17:19 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (5 responses)
Posted Jan 15, 2014 17:24 UTC (Wed)
by paulj (subscriber, #341)
[Link] (4 responses)
The OSS → ALSA transition was fairly pain-free. ALSA → Pulse not so. IME.
Posted Jan 15, 2014 18:01 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
Posted Jan 15, 2014 18:06 UTC (Wed)
by paulj (subscriber, #341)
[Link] (2 responses)
# ls /dev/dsp*
Posted Jan 16, 2014 13:18 UTC (Thu)
by cortana (subscriber, #24596)
[Link] (1 responses)
The major shortcoming with the snd-pcm-oss module is precisely the problem you observe; unless your sound card has hardware mixing, the kernel will only permit one process to open either /dev/snd/pcm/pcm* or /dev/dsp at a time. The last such card I had the displeasure to use was a SoundBlaster Live! almost a decade ago.
You should look in to the CUSE solution (Debian package: osspd) if you still need /dev/dsp; if it's not available for your distro and you don't feel like packaging it yourself then you can always run your OSS-using program via pasuspender, which politely tells pulseaudio to get out of the way while your program runs.
Posted Jan 16, 2014 22:13 UTC (Thu)
by ms-tg (subscriber, #89231)
[Link]
Posted Jan 15, 2014 18:00 UTC (Wed)
by mathstuf (subscriber, #69389)
[Link] (5 responses)
I don't know if there's a way to get OSS programs to use PulseAudio. If not, kill PulseAudio and use the CPC emulator then. PulseAudio tends to get started on-demand (and will block if the emulator is running), so getting back to normal is implict there.
Posted Jan 15, 2014 18:11 UTC (Wed)
by paulj (subscriber, #341)
[Link] (4 responses)
Posted Jan 15, 2014 19:14 UTC (Wed)
by nybble41 (subscriber, #55106)
[Link] (1 responses)
[1] "Character devices in user space" <http://lwn.net/Articles/308445/>
Posted Jan 16, 2014 9:32 UTC (Thu)
by paulj (subscriber, #341)
[Link]
Thanks!
Posted Jan 16, 2014 16:00 UTC (Thu)
by ThinkRob (guest, #64513)
[Link] (1 responses)
Posted Jan 16, 2014 18:33 UTC (Thu)
by pizza (subscriber, #46)
[Link]
...and OSS4 wasn't free, in both meanings of the term.
...that is, until long after ALSA had come to dominate.
Posted Jan 15, 2014 18:54 UTC (Wed)
by smurf (subscriber, #17840)
[Link] (1 responses)
Did you try the padsp wrapper?
Posted Jan 15, 2014 20:25 UTC (Wed)
by nix (subscriber, #2304)
[Link]
Posted Jan 25, 2014 0:01 UTC (Sat)
by makomk (guest, #51493)
[Link] (8 responses)
Likewise, flat volumes breaks in random and amusing new ways with every version, and I've never encountered any of the mythical hardware it actually works on without annoying pops and glitches and random bursts of excessively loud audio when an app starts or exits. Hell, I've even tried it on hardware with no hardware volume control at all and somehow it still managed to screw that up by trying to adjust the non-existent hardware volume control - so lowering the application volume level had no effect until suddenly it muted the output entirely. So far as I can tell, the entire feature relies on hardware behaving in ways that isn't part of any specification and cannot ever be relied on. Though at least that can be disabled if you edit the right obscure configuration file as root.
Posted Jan 25, 2014 1:07 UTC (Sat)
by pizza (subscriber, #46)
[Link] (6 responses)
I counter your anectdote with one of my own; half a dozen or so systems over as many years, all working without an audio-related hitch. All using bog-standard onboard AC97 or HD Audio codecs.
So, what applications are having problems? What were already streaming? What pulseaudio version? What hardware? Have you filed any bug reports?
Posted Jan 27, 2014 22:50 UTC (Mon)
by nix (subscriber, #2304)
[Link] (5 responses)
Posted Jan 28, 2014 1:22 UTC (Tue)
by paulj (subscriber, #341)
[Link]
Posted Jan 28, 2014 2:19 UTC (Tue)
by mathstuf (subscriber, #69389)
[Link] (3 responses)
> flat-volumes = no
Try and see if that fixes it.
Posted Jan 29, 2014 23:35 UTC (Wed)
by nix (subscriber, #2304)
[Link] (2 responses)
Posted Jan 29, 2014 23:55 UTC (Wed)
by mathstuf (subscriber, #69389)
[Link] (1 responses)
Posted Jan 30, 2014 0:49 UTC (Thu)
by nix (subscriber, #2304)
[Link]
Posted Jan 25, 2014 4:18 UTC (Sat)
by raven667 (subscriber, #5198)
[Link]
Posted Jan 21, 2014 22:10 UTC (Tue)
by pgoetz (subscriber, #4931)
[Link]
Posted Jan 23, 2014 18:06 UTC (Thu)
by Wol (subscriber, #4433)
[Link] (6 responses)
"Be conservative in what you send, and liberal with what you accept" just doesn't fit his way of working. If the spec says "when you do X, then Y should happen" then Lennart assumes that Y will happen. Far too much software is far too accepting when Y doesn't happen - Lennart just says "your software is broken, FIX IT!"
I gather that's why PulseAudio was such a mess when it first appeared - it kept tickling *other* *peoples* bugs. And Lennart insisted they fix their bugs, rather than him work around them.
Looked at at a technical level, that's the only approach that makes sense. But it does rather ruffle a lot of feathers.
That seems to have been one major reason for writing SystemD - SysVInit just contains too many design flaws to be even fixable ...
Cheers,
Posted Jan 24, 2014 12:55 UTC (Fri)
by josh (subscriber, #17465)
[Link] (5 responses)
Posted Jan 24, 2014 13:40 UTC (Fri)
by anselm (subscriber, #2796)
[Link]
Working around other people's bugs basically gets you HTML. You have to figure out everybody else's workarounds to put those in your code, too, and it never stops.
Posted Jan 24, 2014 19:26 UTC (Fri)
by dlang (guest, #313)
[Link] (3 responses)
In theory you are correct, in practice being intolerant of bugs in other people's code just doesn't work very well and causes your users a LOT of pain.
unless you are really able to force your users to use your code even when it's a worse experience than what you are replacing, (by getting a major distro to remove the old option and ship yours for example), you are really not likely to gain a lot of users. You will definitely earn your users ire and distrust. This distrust will take a LONG time to go away, if it ever does.
As this subthread is showing with the complaints years later about PulseAudio
Posted Jan 24, 2014 20:33 UTC (Fri)
by smurf (subscriber, #17840)
[Link]
Plus, you really cannot blame the distros. The features pulseaudio offered were too compelling. Remember that there basically was no hotplugging of audio devices at the time. Today I can pair up my Bluetooth headphones and immediately switch over my background soundtrack without even hearing an audio glitch.
Likewise, some of the things systemd does are somewhat annoying, but once you start taking even a third of its features for granted you won't ever switch back. (IMHO.) I suspect it'll be much the same with kdbus.
Posted Jan 24, 2014 20:38 UTC (Fri)
by anselm (subscriber, #2796)
[Link]
»Be strict when you send and tolerant when you receive« wasn't such a great idea in the first place. It serves more as an excuse for sloppy implementations than it does improve interaction between programs. It is better to be just as strict when you're receiving stuff as when you're sending stuff so bugs can be found earlier. If everybody is being tolerant then the sloppy senders never get fixed, and if you write new receiving code to the specs you will have to figure out all the workarounds that are required to handle the entrenched sloppiness. (And of course the workarounds will be different in different implementations such that sloppy stuff leads to different results.) This sucks but fixing it can be an uphill battle – for example, the W3C came up with XHTML to get rid of the ingrown sloppiness in HTML, but we all know what became of that idea.
It is tough being the only strict person when all around you everyone is complacent about sloppy code – even if your code is a lot better, people won't like it if their stuff breaks, and they will make you the culprit even if they themselves are technically at fault. Nobody likes a stickler for the rules because those make their life more difficult. But that doesn't change the fact that the rules are there for a reason.
Posted Jan 24, 2014 21:19 UTC (Fri)
by khim (subscriber, #9252)
[Link]
Not software but a specification: XML. Violations of well-formedness constraints are fatal errors which means when you try to feed non-well-formed document to an application usually you see flat-out rejection. Works pretty damn well in practice.
Posted Jan 13, 2014 22:55 UTC (Mon)
by error27 (subscriber, #8346)
[Link]
Posted Jan 14, 2014 1:10 UTC (Tue)
by dw (subscriber, #12017)
[Link] (7 responses)
There is little preventing an ABI-compatible libdbus that does little but coordinate a directory held in shared memory or similarly stashed elsewhere, having just enough logic to setup mappings or socket pairs between peers, with all validation done by the library code at the receiver. There is even no requirement for the filesystem to be present thanks to e.g. the SO_UNIX abstract namespace that's been around since forever.
For similar reasons I've never understood the value of the DBUS daemon – does it manage some super secret policy that its clients aren't allowed to peruse directly?
Reminds me a lot of devfs
Posted Jan 14, 2014 1:40 UTC (Tue)
by smurf (subscriber, #17840)
[Link]
I strongly suspect that by the time you implemented that, then write the userspace library to coordinate everything so that it presents roughly the same interface as userspace dbus, you'd have more code with less security than what kdbus will offer.
For instance, proper message marshalling can be checked once - by the kernel - or by every recipient. Guess which is going to be more efficient.
Posted Jan 14, 2014 2:49 UTC (Tue)
by hp (guest, #5220)
[Link]
Not everything called "IPC" is the same thing with the same properties and requirements.
Posted Jan 14, 2014 3:36 UTC (Tue)
by mezcalero (subscriber, #45103)
[Link] (4 responses)
The difference between kdbus and dbus-daemon otoh are: performance, support for exchanging extensive payload not just control, timestamping, more credentials, kernel-side hookup with LSMs, race-free deactivation for services, ... -- The big one however is certainly the first mentioned: performance.
Lennart
Posted Jan 14, 2014 7:50 UTC (Tue)
by edomaur (subscriber, #14520)
[Link] (2 responses)
I know that's not your problem and I even agree to your point of view regarding other platforms, but desktop environments are not for Linux only, and D-BUS must be working the same way everywhere it is ported. How kdbus is working and how its new protocol addons must be documented in a way abstracted from the code.
(Anyway, I'm not a coder anymore, so I will not take the task.)
Posted Jan 15, 2014 9:18 UTC (Wed)
by pbonzini (subscriber, #60935)
[Link]
Posted Jan 15, 2014 19:29 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link]
So, there's basically going to be a dbus "baseline", which includes certain features and perfomance expectations. If people stick to that (and a lot probably will) then things stay perfectly portable.
Posted Jan 14, 2014 11:58 UTC (Tue)
by bokr (guest, #58369)
[Link]
E.g., imagine that you have built a signed (by yourself) UEFI-bootable kernel that can serve as a jailhouse[1] hypervisor.
Then imagine that you can trigger a Secure Attention signal by pushing a button with tamper-evident direct path to a NMI that the hypervisor sees. (Ok, for now, trust the hypervisor kernel to detect Ctl-Alt-Del in kb input ;-)
Imagine that the hypervisor loads a signed trusted login process image into a separate jailhouse, and you log in. Now you are in a user shell sort of like bash after Alt-F2 plus login, only you are a self-contained booted hypervisor guest process in a jailhouse cell.
How would kdbus support secure communication with other on-the-metal guests in other cells? I am visualizing a truly paranoid system where all apps and kernel services are signed bootable monolith images that need to communicate securely.
Posted Jan 14, 2014 6:22 UTC (Tue)
by mitchskin (subscriber, #32405)
[Link] (10 responses)
I'm not sure how much sense that makes if you have an async API, though. Context switching on a blocking call seems fairly straightforward, but it's not clear to me how the CPU-yielding would work in the async case.
Posted Jan 14, 2014 6:33 UTC (Tue)
by jonabbey (guest, #2736)
[Link] (1 responses)
Posted Jan 14, 2014 7:31 UTC (Tue)
by wahern (subscriber, #37304)
[Link]
Posted Jan 14, 2014 7:37 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link] (1 responses)
Posted Jan 14, 2014 11:39 UTC (Tue)
by pbonzini (subscriber, #60935)
[Link]
Nowadays, the LPC implementation is still used by some system processes, and under the hood by DCOM RPC when the source and destination reside on the same host.
Posted Jan 14, 2014 16:59 UTC (Tue)
by gregkh (subscriber, #8)
[Link] (4 responses)
Posted Jan 14, 2014 19:56 UTC (Tue)
by smurf (subscriber, #17840)
[Link] (1 responses)
Also, I wonder whether we need it in the first place. Today's phones have multiple CPUs, presumably one of them is idle; besides, the source task will probably go to sleep, waiting for the reply, within a msec or two.
Posted Jan 14, 2014 20:16 UTC (Tue)
by gregkh (subscriber, #8)
[Link]
Patches gladly accepted :)
> Also, I wonder whether we need it in the first place
The Binder interface relies on this process flow for how it works. If you want to, just change the kernel binder code and see if anything breaks in your Android system...
Posted Jan 15, 2014 5:21 UTC (Wed)
by mitchskin (subscriber, #32405)
[Link] (1 responses)
Posted Jan 15, 2014 14:48 UTC (Wed)
by gregkh (subscriber, #8)
[Link]
And Kay and Lennard really did try to be able to replace binder with kdbus, but everyone agrees, the binder developers included, that this just will not work, so no one involved is unhappy or upset about it at all.
Just remember, never use binder for any code outside of an Android-only system, it is very dangerous to do so, and you get to own the pieces of your kernel if you happen to do so.
Posted Jan 16, 2014 9:54 UTC (Thu)
by renox (guest, #23785)
[Link]
Last time I read about it is here: http://lwn.net/Articles/576691/
> but it's not clear to me how the CPU-yielding would work in the async case.
Posted Jan 14, 2014 8:02 UTC (Tue)
by nhippi (subscriber, #34640)
[Link] (1 responses)
Posted Jan 14, 2014 8:16 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Jan 14, 2014 8:30 UTC (Tue)
by djc (subscriber, #56880)
[Link] (1 responses)
http://www.slideshare.net/jamesskillsmatter/martin-sustri...
Posted Jan 14, 2014 10:55 UTC (Tue)
by ms (subscriber, #41272)
[Link]
Posted Jan 14, 2014 13:43 UTC (Tue)
by jspaleta (subscriber, #50639)
[Link] (11 responses)
So reading through the thread in full, and I'm only seeing one really interesting point of contention. It appears that the app sandbox model being used for Unity treads into a bit of a grey area with regard to security policy enforcement. The functionality they are relying on appears to be an implementation detail of dbus-daemon and not part of the D-Bus spec which all daemons are expected to provide.
Up till now its not really come up, because we haven't had a second implementation of the spec for the "may" clause to come into effect. But now, kdbus as a second implementation of the spec, doesn't include that optional security policy check. It doesn't have to, as the D-Bus spec allows this to be an implementation by implementation choice to make.
And so the concern is, the sandbox model Ubuntu is using is layered over dbus-daemon in Ubuntu isn't going to work with kdbus as designed. Noone is at fault here. Decisions were made to rely on optional implementation functionality not covered by the protocal spec. And the kdbus design decisions preclude the optional security policy check.
The only salient question is this. Will Ubuntu's sandbox model continue to require the reference dbus-daemon implementation, or can modifications be done to either Ubuntu's user space or to kdbus design so Ubuntu can take advantage of kdbus performance benefits in later releases? The irony of course is that Ubuntu's mobile push would probably benefit the most from kdbus's performance gains, but at the same time its also where their sandbox model is most useful as well.
-jef
Posted Jan 14, 2014 15:41 UTC (Tue)
by paulj (subscriber, #341)
[Link] (10 responses)
Posted Jan 14, 2014 16:56 UTC (Tue)
by ovitters (guest, #27950)
[Link] (9 responses)
Also quite interesting is the difference between the how Ubuntu app sandboxing will work and the one envisioned for kdbus/gnome/whatever to call it. You'll note that both sides tried to work together, though didn't pan out in the end.
Then you have tidbits about Samsung already using an old version of kdbus. Further, that kdbus won't be used easily by Binder (the Android thing). I thought it would be able to be used for Binder as well, so interesting to read. It makes me wonder what Samsung is using it for (guessing Tizen, but would be nice to know for sure).
Posted Jan 14, 2014 19:25 UTC (Tue)
by smurf (subscriber, #17840)
[Link]
As far as I understand, if somebody (Ted Gould in this case) says to Lennart that
>> I see you as delivering an ultimatum to the DBus community about how they've been replaced.
then I guess I can understand why Lennart doesn't want to continue that kind of "discussion". The responsibility to keep a discussion civil and factual is shared by both sides of a debate; blaming Lennart for that subthread's ultimate failure to commuicate is unfair.
Posted Jan 14, 2014 21:14 UTC (Tue)
by tanuk (guest, #71154)
[Link]
Yes, Tizen, judging from this message (Tizen git repo referenced): http://thread.gmane.org/gmane.comp.freedesktop.dbus/15499...
Posted Jan 15, 2014 11:31 UTC (Wed)
by jspaleta (subscriber, #50639)
[Link] (6 responses)
Here's a good article from May 2013 to recap the dev thinking on this from the kernel-side: http://lwn.net/Articles/551969/
Note the brief discussion about Binder's security issues. I think that's very interesting. I would think Google would want in the long term to have both ChromeOS and Android being able to use the same IPC securely. kdbus might be that long term solution for them, on some timescale. But who knows.. Google will cross that bridge or not in due time. The point I think is a transition to kdbus from binder is not out of the question for Google. Google won't have to completely changing the design of their intents system.
-jef
Posted Jan 15, 2014 19:24 UTC (Wed)
by mezcalero (subscriber, #45103)
[Link]
Binder doesn't really queue things so much, it's synchronous by default, and its backend is dispatched by worker threads (i.e. fully unordered), while dbus is more focussed on async stuff, and gives global ordering guarantees.
Maybe some lower-level work can be shared between binder and kdbus, but kdbus itself is not suitable as a backend for binder, and won't be.
Lennart
Posted Jan 15, 2014 23:04 UTC (Wed)
by ovitters (guest, #27950)
[Link] (4 responses)
Posted Jan 15, 2014 23:12 UTC (Wed)
by jspaleta (subscriber, #50639)
[Link] (3 responses)
-jef
Posted Jan 16, 2014 10:18 UTC (Thu)
by sdalley (subscriber, #18550)
[Link] (2 responses)
Any nice pix you'd fancy sharing? What do they do for internet service down there?
Just curious.
Wishing you a pleasant summer holiday.
Simon
Posted Jan 16, 2014 10:48 UTC (Thu)
by jspaleta (subscriber, #50639)
[Link] (1 responses)
Posted Jan 16, 2014 15:37 UTC (Thu)
by raven667 (subscriber, #5198)
[Link]
Ha ha, that explains why you are here so much but what excuse do the rest of us have 8-)
Posted Jan 14, 2014 20:04 UTC (Tue)
by jonabbey (guest, #2736)
[Link]
The video is up now:
Posted Jan 16, 2014 11:20 UTC (Thu)
by lainfinity (guest, #94972)
[Link] (2 responses)
Linux is Linux because of its monolithic architecture, even though less secure Linux microkernel is no longer Linux may be it will eventually be transformed to Lennarix.
Android vs GNU/Linux even though both is based on Linux kernel, there is a big difference in terms of stability. I prefer a less secure and stable monolithic than more secure microkernel.
History always repeats (Monolithic vs Microkernel).
Posted Jan 16, 2014 12:47 UTC (Thu)
by alexl (subscriber, #19068)
[Link] (1 responses)
Posted Jan 16, 2014 14:41 UTC (Thu)
by lainfinity (guest, #94972)
[Link]
http://kroah.com/log/blog/2014/01/15/kdbus-details/
Eventually because of the compexity and functional capabilities of the IPC like kdbus Linux will tend to behave more like a microkernel in the long run.
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
[2]Seems to mean "committed upstream"; not in Fedora (yet?).
[3]Just going by the summary for the most part.
The unveiling of kdbus
memfds
memfds
memfds
memfds
memfds
memfds
memfds
memfds
Which is why we have HTTP, and no other way for applications to manipulate TCP/IP sockets, because that would be reinventing the wheel.
</snark>
memfds
memfds
With regard to AF_DBUS, they have addressed that problem by avoiding the network stack altogether. The functionality is fully contained within a char device now.
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
Oh yes, (e) it should have enough features to eventually supersede Android's binder and ashmem drivers.
Having no information to the contrary and being aware of the complexity of the problem, I tend to agree.
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
AF_DBUS
memfds
"Remember the HTTP server wars? Microsoft had a blazing fast HTTP server in-kernel. So somebody coded one up in the Linux kernel, to one up Microsoft. But then somebody with infinitely more sense tweaked some kernel APIs and wrote an HTTP server in user-space that bested both the in-kernel servers."
History: In-Kernel HTTP Acceleration
History: In-Kernel HTTP Acceleration
History: In-Kernel HTTP Acceleration
Nobody "called TUX a pinnacle of engineering".
The only thing the GPP stated was the true fact that it outperformed other web servers by an orders-of-magnitude margin.
History: In-Kernel HTTP Acceleration
History: In-Kernel HTTP Acceleration
memfds
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
then the cost of moving a few metadata headers to the benefit of all lsm is miniscule. a central point of control is the nature of security, devolving all this to untrusted apps is inane
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
> unpleasant ideologues use to dismiss people who won't go along with their
> ideology.
The unveiling of kdbus
Power users can easily create another login if they need a separate session for whatever reason.
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
Ubuntu had not backported a kernel patch that would allow rtkit to run properly, so PA was unable to get realtime priority.
This HOWTO suggests that they didn't divert ALSA apps to use PA, which meant those apps' audio would not be mixed with the audio from PA-using apps.
The unveiling of kdbus
Like what?
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
- he couldn't even predict there'd be any problems!
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
/dev/dsp /dev/dsp1
# lsmod |grep oss
snd_seq_oss 38268 0
snd_seq_midi_event 14437 1 snd_seq_oss
snd_pcm_oss 53613 0
<etc>
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
[2] <https://fedoraproject.org/wiki/Features/OSSProxy>
The unveiling of kdbus
The unveiling of kdbus
Yes indeed, OSS only ever supported one user.
Just as clarification: Linux's built-in implementation of OSS only ever supported one user. Other implementations (such as 4Front's OSS4 for Linux) did and do support multiple concurrent applications producing sound.
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
Wol
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
what software that is completely intolerant of any bugs in other implementations has every become significant?
unless you are really able to force your users to use your code even when it's a worse experience than what you are replacing, (by getting a major distro to remove the old option and ship yours for example), you are really not likely to gain a lot of users. You will definitely earn your users ire and distrust. This distrust will take a LONG time to go away, if it ever does.
I think you've just justified Lennart's position. If you introduce a new software which does not tolerate mistakes (e.g. pulseaudio or expat) then people may adopt it or refuse it but they will be forced to keep everything straightened out. If you'll introduce “temporary” workarounds then you'll be forced to carry them more-or-less forever.The unveiling of kdbus
Why?
Why?
Why?
Why?
Why?
Why?
Why?
Why?
[1] http://lwn.net/Articles/574273/
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
Yielding CPU time to destination
It wouldn't, otherwise it's not async.. That said, if you fill the buffer an async IPC "degenerate to synchronous" (if you see what I mean), so it could still be interesting to yield the CPU if/when this happen.
kdbus, for dbus only?
kdbus, for dbus only?
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
Quoting the spec:
http://dbus.freedesktop.org/doc/dbus-specification.html
"Message bus implementations may impose a security policy which prevents certain messages from being sent or received"
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
Waiting for Da BUS at South Pole
Waiting for Da BUS at South Pole
Waiting for Da BUS at South Pole
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus
The unveiling of kdbus