| |||||||||||||
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernelKroah-Hartman: AF_BUS, D-Bus, and the Linux kernelPosted Feb 8, 2013 23:05 UTC (Fri) by brouhaha (guest, #1698)In reply to: Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel by kugel Parent article: Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Maybe we can move Firefox and LibreOffice into the kernel next. Then we can do away with user space entirely.
(Log in to post comments)
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 9, 2013 0:08 UTC (Sat) by BrucePerens (guest, #2510) [Link] I know you're being sarcastic, but I'd take the collection around udev and move it back into the kernel first.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 9, 2013 2:55 UTC (Sat) by brouhaha (guest, #1698) [Link]
Supposing that this D-Bus stuff was already in the kernel, and someone came up with a clever way by which adding a new address family to the networking code would allow moving D-Bus out of the kernel, I'd be very much in favor of moving it out.
Obviously the same reasoning should apply to the case where D-Bus is NOT already in the kernel.
In general, I'm in favor of moving things out of the kernel. For example, I think putting KMS and DRI in the kernel were steps in the wrong direction.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 9, 2013 15:54 UTC (Sat) by raven667 (subscriber, #5198) [Link]
Having the reliable multicast transport for local data grams seems like it should be an uncontroversial good feature, and it probably would be if the biggest consumer of it wasn't called d-bus. I don't know how IPC became so politicized.
As far as KMS and DRI, they are definitely in the right place, keep the hardware management in the kernel and all the complicated graphics stack in userspace.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 10, 2013 4:49 UTC (Sun) by bronson (subscriber, #4806) [Link]
No worries, anybody who thinks that modesetting belongs in user space can't possibly happy with the monolithic Linux kernel anyway. These are uninformed objections, probably written without having looked at any of the code first. Can't imagine they'd get any traction among kernel devs.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 12, 2013 10:05 UTC (Tue) by ortalo (subscriber, #4654) [Link]
I agree with you. But I am not a kernel devs, and it took several years (I'd say 1998-2004) for the kernel devs to agree fully for KMS in the kernel... Fortunately, these days are gone but the devs are probably still there (fortunately too! ;-).
BTW: Note that I evolved too and now also have sympathy for the idea of not bothering *at all* with undocumented hardware when doing kernel programming. I certainly wouldn't have admitted that 10 years ago, but GPUs nearly fit that class.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 9, 2013 17:03 UTC (Sat) by quanstro (guest, #77996) [Link]
it's interesting to me to compare the plan 9 plumber to D-BUS. the plumber
is a dynamic uni/multicast router built from a regular 9p file server.
at a very high level they seem to fill similar roles. (i know there's a lot
with this in mind, if going to the trouble of reworking d-bus, why not think
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 9, 2013 18:40 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]
Because it's retarded and stupid.
DBUS is not in the business of sending/sharing files.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 10, 2013 15:02 UTC (Sun) by quanstro (guest, #77996) [Link]
this sort of response misses the point, and misunderstands the mechanism.
the plumber doesn't pass the file around, it passes a message around, usually with a pointer (path) to the file. here's a direct link http://www.plan9.bell-labs.com/magic/man2html/4/plumber the literal plumber however isn't even the point. the point is that plumber is an example of a file server which routes messages.
also, there is a long tradition in plan 9 of using file servers for messaging.
whatever happened to the idea that "everything is a file"?
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 10, 2013 17:04 UTC (Sun) by hitmark (guest, #34609) [Link]
People either took it too literary, or didn't get the performance they wanted to they fell back to passing memory address ranges back and forth.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 10, 2013 4:51 UTC (Sun) by bronson (subscriber, #4806) [Link]
Because the plumber is based on regexes with almost no prescribed policy. It sure looks like an unscalable maintenance nightmare to me. But, I admit, I haven't used it myself... Curious if my worries are unfounded.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 10, 2013 14:33 UTC (Sun) by quanstro (guest, #77996) [Link]
don't get me wrong, i wasn't saying the plumber solved this problem, i
just think it solves a similar problem. and it's worth considering if some of the mechanism might be right for this problem.
there are a few reasons this is never a problem on plan 9
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 4:20 UTC (Thu) by mmarq (guest, #2332) [Link]
"" In general, I'm in favor of moving things out of the kernel. For example, I think putting KMS and DRI in the kernel were steps in the wrong direction. ""
Why care to elaborate ?
In my limited view it seems to have stabilized the things above in the display drivers arena.
To me it seems clear that the trend in display/graphics is GPGPU... even for Intel (sooner or later)... and without wanting to offend feelings but the leader of the client side (ARM) is heavy on the HSA, along with the heavy weight Samsung, and their specification even includes the GPGPU as bus master in a boot process... something that shouldn't be strange to Intel since the PCIe v3 spec they control has provisions for it with the new multiplexing protocol of HP origin that they included.
I don't see any CPU low level driver or interface in userspace (if wrong correct please)... why should it be any different for GPGPU, that in HSA is not to be only because of name or workload, but to be a "processor" not a "device" ??
I think Dbus in kernel space could find many uses.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 4:51 UTC (Thu) by brouhaha (guest, #1698) [Link] The only thing that the DRI and KMS code running in kernel space is doing that couldn't be done perfectly well from user space is handle interrupts. It should be entirely adequate to have a very small driver in kernel space that does nothing but allow a user-space thread to block until the graphics card requests an interrupt.There would still have to be different kernel drivers for different families of graphics cards, but they would all be very small.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 5:01 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
EVERYTHING could be done from user space. See: QNX, Mach, GNU HURD.
It doesn't make it a good idea. Generally, everything that touches hardware directly should live in kernel.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 5:14 UTC (Thu) by brouhaha (guest, #1698) [Link] EVERYTHING could be done from user space. [...] It doesn't make it a good idea.Nor does it necessarily make it a bad idea. EVERYTHING could be done from kernel space. It doesn't make it a good idea. You need a better justification to support putting stuff in the kernel. Generally, everything that touches hardware directly should live in kernel.Aside from that having been historically true, back when hardware was MUCH simpler, why should that be a particularly important distinction today? Back then we didn't have mmap() and such things, but now that we do, there's a lot less justification for throwing things into the kernel just because they happen to touch hardware registers. Not too many years ago I worked for a very large router company, and in several of our Linux-based product lines, we put nearly everything that touched the hardware in user space, other than (as I suggested for graphics drivers) small kernel-space drivers to reflect interrupts to user space. Our research determined that there was no significant performance benefit to having our hardware drivers in kernel space, while there were significant advantages in maintainability, error recovery, and general robustness to having that driver code be in user space.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 5:27 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
How do you do security, synchronization (including protection from malicious actors), command submission, VT handoff and other stuff from the userspace? Oh, and also hotplug, DMA buffer sharing, interaction with V4L and other nice stuff.
Userspace GPU drivers never performed well. And ability to handle interrupts was the least of the problems - most real problems were in trying to choreograph the complicated dance of device handoffs between BIOS, kernel framebuffers and userspace GPUs.
Besides, all this shiny new GPU infrastructure allows to kill off stuff that REALLY should not be in the kernel: VT102 emulator for kernel framebuffers.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 5:48 UTC (Thu) by brouhaha (guest, #1698) [Link] I'm not sure about VT handoff or interaction with V4L, but none of the other features that you've mentioned are especially better due to being in the kernel. There are suitable mechanisms for all of them for user-space processes. If there weren't, we'd invent some.If VT handoff or interaction with V4L have some particularly tricky requirements, then perhaps doing those from userspace might need some additional infrastructure. I'm not saying that I think we should go back to exactly the same userspace graphics code we used to have. There were definitely things wrong with those. However, moving chunks of it into the kernel isn't the only way to solve the problems we had. Userspace GPU drivers never performed well.That might be true, though I never personally had any significant performance issues with them. But even if it was true, I don't think there's anything inherent to the problem that prevents it from being done in userspace with performance comparable to what we get in kernel space. That's certainly what we found at the very big router company. There's nothing magic about being in kernel space that makes code run significantly faster. Certainly in user space you need to avoid doing things that involve copying memory from one process address space into kernel space then again into another process address space. Naively designed code tends to do a lot of that, but it's quite possible and not even that difficult to avoid it by using modern OS facilities. Besides, all this shiny new GPU infrastructure allows to kill off stuff that REALLY should not be in the kernel: VT102 emulator for kernel framebuffers.I'd be perfectly happy to have neither in the kernel, but if I had to choose between one or the other, I'd definitely keep the relatively small, simple, lightweight VT102 emulator in the kernel and put the big, complicated GPU stuff in user space. I never expected anyone to take my comments about this very seriously. I don't have the time or inclination to work on GPU drivers. The people that actually do the work get to make the technical decisions about it, and I wouldn't have it any other way. This is just an exercise in being an armchair quarterback.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 5:54 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
> There are suitable mechanisms for all of them for user-space processes. If there weren't, we'd invent some.
Please, go and educate yourself. GPU interaction requires an always-on privileged process that should be able to talk directly with hardware. If this process crashes you can easily get a hard system lockup.
Oh, and this process must be able to get _exclusive_ access to the hardware. Because it intrinsically can't be shared.
Does it start to sound familiar? No?
> I'd be perfectly happy to have neither in the kernel, but if I had to choose between one or the other, I'd definitely keep the relatively small, simple, lightweight VT102 emulator in the kernel and put the big, complicated GPU stuff in user space.
The kernel-side drivers basically manage video buffer allocation and command submission. I.e. they take buffers with command batches and send them to hardware. They don't do anything that is fundamentally complicated.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 6:15 UTC (Thu) by brouhaha (guest, #1698) [Link] If this process crashes you can easily get a hard system lockup.So? If a kernel driver crashs, you can easily get a hard system lockup. Been there, done that, got the T-shirt. I've been device driver developer on various flavors of Unix for more than 20 years. Putting code in user space doesn't magically make it less robust, any more than it magically slows it down. If you want robust code, you have to design it well, regardless of whether it's in kernel space or user space. Have you actually SEEN the GPU drivers?As a matter of fact, I have looked at the GPU drivers, and they seem a lot more complicated than the VT102 emulator to me.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 8:47 UTC (Thu) by mmarq (guest, #2332) [Link]
well i'm not "device" driver developer... and what you say seems more or less reasonable for a device...
i suggest http://hsafoundation.com/ a tour by the specs and docs...
quite a list and quite heavy weights no ?
So what seems to stand out against your arguing in my view... even without deep tech arguments that i don't have now(if i ever)... is that is NOT about "devices" anymore but "processores"... yes talk GPU talk processor...
And none of other contender to this spec will be out... matter of fact Intel already has big "physics" cards... Apple neither will be in the shadow... all this things are/will be **fully programmable**.
It is a wonder some news about gaming in Linux... when some of its developers seem to want to keep it 3th class in the sector... boy! it is a luck to have game studios interested...
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 8:13 UTC (Thu) by mmarq (guest, #2332) [Link]
"" I'm not saying that I think we should go back to exactly the same userspace graphics code we used to have. ""
Uff!... thank god! lol
I want the tremendous *compute power* of the GPGPU ... i want the ability to do "co-designed" on-the-fly binary translation with profiling for that compute power... i want all my graphics ray-traced for illumination, with OpenCL style physics effects ( hey! why not a KDE or Gnome environment with "physics" effects and animations ?... not only games ??)
quite a list no!? ... i think doing all that from userspace would make programing Cell look like superball.. lol
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 18:01 UTC (Thu) by nix (subscriber, #2304) [Link]
Actually almost all of the computation side of things *is* done in userspace. It would be madness to put LLVM into the kernel!
But things like physical memory management and mediation between competing users -- that's what kernels *do*. It's what they do for the realm the CPUs rule over, and it seems perfectly sensible to have them do it for the GPU's realm too (and a heck of a lot more robust than the alternative).
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 19:39 UTC (Thu) by mmarq (guest, #2332) [Link]
Don't follow, who said about puttin LLVM into the kernel ?
If traces parallels with Android model, yes they might adopt something like Aparapi (translates Java into OpenCL like, with knowledge of target ISAs) and push for a backend compiler or "finalizer" into the kernel.
But this is quite smaller than LLVM and resides below LLVM...quite low level.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 15, 2013 18:15 UTC (Fri) by nix (subscriber, #2304) [Link] Don't follow, who said about puttin LLVM into the kernel ?You did. The 'tremendous compute power of the GPGPU' is implemented by compiling things targetted to the GPU, via LLVM (at least that's the lion's share of it). Putting that in the kernel means putting LLVM in the kernel.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 4:21 UTC (Mon) by mmarq (guest, #2332) [Link]
he!..wrong.
I'll try in lay terms (warning: NOT a developer of this)
The HSA standard if that is what we are following, doesn't need kernel inclusions... i'm confident... not even the runtime... only the "finalizer" that touches metal directly could induce the need.
This back-end compiler is very ISA specific, is its job->knowing well the target and optimize for it with runtime info, its a JIT compiler, things that static compilers cannot do(well), even LLVM that sits above;
So most probably will be several different "finalizers" for the many "processores" HSA encompasses, including not only ARM/AMD/Imagination GPUs but also TI DSPs etc.
So it could be a DKMS feature, if Linux supports DKMS properly(?)...
Matter of fact it doesn't even have to be a "finalizer", the HSA runtime can take care of that like the OpenCL runtime or the C++11/AMP runtime does.
For the more performance aware version with a "finalizer", the only features that touch hardware are DMA(direct memory access), VM stuff, TLB stuff, scheduling and IOMMU stuff... things that seem not indicate GPU at all no ?...
And it could be done by improving what is already there... i don't think the patches must be intrusive if it comes to that, since the hardware features listed on the spec are really very small.
Matter of fact the basic spec seems like any other language, with its proper runtime... and i don't see the need for C or C++ libs on the kernel(?)... neither should this, tough its idiom HSAIL, is very low level and compatible with LLVM IL SSA, that is, it takes SSA transforms it to HSAIL then native by a "finalizer", this if you want your program having the same Virtual Memory space, meaning CPU + GPU/DSP/etc will have the same space for better integration overall... then the "finalizer" is the way to go...
OTOH it can also proceed directly from the IL to the target by the runtime, and a program can be build with both targets by their tools, native + HSAIL(finalizer), using exactly the same code, so even if Intel NEVER implements the hardware features, any HSA program could run on any Intel CPU *platform*(so far) **UNCHANGED**...
Only with the "finalizer" stuff, the program can run quicker in perhaps most cases(if properly coded)(less heavy or less specific stuff and suspect no difference at all)...
The idea is not only use the same *virtual memory space*, is use ANY language to program it... and much better than nVidia CUDA IMHO... it goes so far to support, C/C++, AMP, OpenCL, fortran Java(+scripts) Python and more, i think more than the supported natively by LLVM...
The advantage with a "common virtual space" is that any GPU/DSP/etc memory will be managed as system memory not a different pool... also the GPU/DSP/etc could have its context/exception handling and interrupt handling mostly outside of any CPU control (like if it were another CPU), which is better for power management... its a UMA heterogeneous SMP architecture, an obvious very good evolution to the co-processor idea...
intros
The drawback with this is i don't know if its compatible with the current DRM implementation, or if it can be patched to it, since rendering seems only one of the features that encompasses the standard, it could be complementary... or perhaps!... Intel top management, namely its CEO gets one of those Christmas Ghost moments, and Intel joins the HSA... eh!... without need for false promises to the ghost lol... since Intel doesn't need to license anything (i think), it only have to make its hardware features compatible -> then there will be no bickering but a patch flood... lol
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 4:28 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
Basically, yet another pie-in-the-sky project from purely academic guys without real hardware?
Yeah, it will immediately beat all other architectures, sure.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 5:07 UTC (Mon) by mmarq (guest, #2332) [Link]
Samsung, ARM, TI, Sony, AMD, Imagination, LG, STMicro, ST Ericsson etc etc etc... are academic guys without real hardware ?
Sorry but you are babbling nonsense... without ever having toke a pick on anything...
Only Samsung itself is bigger than Intel, and its not only IT stuff, Samsung and LG makes perhaps the majority of the consumer electronics world...
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 5:23 UTC (Mon) by mmarq (guest, #2332) [Link]
oh! forgot
*Sony*, Samsung, LG... IT IS *THE* consumer electronics world...
You better say "i don't want Linux on those things"... hard task to fulfill no doubt... and the new Playstation IV, you also don't want to see it with a Linux OS ??... doubt they will ever license Microsoft, or (M$ would accept)...
Well, PS IV is going to be HSA modeled... is not PowerPC, is x86 AMD, and Sony entered the HSA... the academic guys grow up and made real hardware lol ... and a flush of shills entered the Linux forums, or is that veterans went to sleep... or are pesky politics abounding ... ummm !!?..
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 19, 2013 19:25 UTC (Tue) by khim (subscriber, #9252) [Link] Well, PS IV is going to be HSA modeled... is not PowerPC, is x86 AMD, and Sony entered the HSA... Wow! Looks like SONY just can't ever produce nice, easy-to-program console. After Cell fiasco (which is theoretically so much more powerful then XBox's CPU but so hard to program that most PS3 game ports are inferior to XBox360) then decided to use of-the-shelf-hardware but now they want to still screw the developers in some other way? I wish them luck, they'll need it.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 19, 2013 22:50 UTC (Tue) by mmarq (guest, #2332) [Link]
Funny you say that, because Microsoft and Nintendo, decided exactly the same thing... so by your POV their are all up to screw the developers! lolzz...
Sony should not be about nice and easy to program... it would be nice for a change... but primarily Sonny i think is after "PERFORMANCE"(try to match both the why of HSA)
See!?... the reason is simple, many bickering in hardware/review sites... but its seems CLEAR that if you want performance, PERFORMANCE IS IN THE SOFTWARE... arguing this CPU and that processor is best than this or that, only controversy to entertain morons!... it is best but with what software ???
More!... according to a leak that circulates, the Durango GPGPU that goes for the Xbox720 is very unusual and seems a highly customized design... many still do laugh and foresee the doom of Xbox, but that is what Microsoft choose, perhaps fitting better their tools and their programming paradigms... since its clear PERFORMANCE IS IN THE SOFTWARE, its highly premature to judge anything only by the hardware features alone...
So your argument of "off-the shelf" is simple WRONG by many accounts... and you should applaud because gives the software side much more leverage.
But i understand your point about "off-the shelf", the "traditional" world of graphics and video is just too stupid to begin with... with super bloated Driver an VM JIT layers for the GLs of the GPGPUs -> crazy!, usually those Console guys **program much more to-the-GPGPU-metal** and are able to extract an **order of magnitude (10x) more performance** than in the "PC world"... Microsoft is in the same boat some how, Xbox and Microsoft will be around *compute* and C++AMP allover...
LINUX IS NOT ABOUT "COMPUTE POWER", AND POWERS SEEMS NOT WANTING IT TO BE... that is why client side "mass adoption" concerning (it has to do with graphics/video)... Microsoft will win for Linux in the foreseeable future, and history repeats itself... even when they F### up as they did with Windows 8...
Sounds crazy but its true... and there is where HSA foundation entered... the idea is exactly "facilitate" this, open it to general languages, and that is why it catched so many interest and is a "de-facto" and "de-jure" standard right now.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 20, 2013 17:54 UTC (Wed) by khim (subscriber, #9252) [Link] usually those Console guys **program much more to-the-GPGPU-metal** and are able to extract an **order of magnitude (10x) more performance** than in the "PC world" This one phrase says it all: bollocks. If you ever actually go and compare console versions of games and PC versions of the same you'll see that yes, they achive "10x more performance"... by replacing nice textures with tesselation and other niceties with blurry POS. Hardly an achievement worthy talking about. I think I'll side with viro: this is not a slashdot and since effective discussion with you is impossible I'll just use this nice LWN's feature and silence you in my stream. Have a nice day.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 20, 2013 22:17 UTC (Wed) by mmarq (guest, #2332) [Link]
hey! the discussions is not me, don't try to make it personal.
I even can agree with some things you say, but everything must be in its proper context, and those are not fixed, they evolve tremendously...
Its not to hurt feelings, to make controversy... but the idea i trowed about forking the Linux Kernel... as exposed... perhaps are not off base.
And i'm not the right person to answer most things, i'm sure the HSA have ppl around here, they could answer erroneous ideas if they are "allowed"... perhaps in another thread...
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 21, 2013 8:11 UTC (Thu) by elanthis (guest, #6227) [Link]
If you compare equivalent hardware, yes, we do make way more use of the hardware in consoles than PCs. This is entirely because we are targeting a fixed hardware spec, though. PC versions can be better because you're comparing 2013 era PC hardware with 2006 era console hardware, which is why mmarq's argument is flawed even if he has a glimmer of insight in his rambling.
There was some AMD buffoon who clearly only knew consoles claiming that D3D12 would likely do away with most of the API since developers wanted to directly target the hardware. This is obviously impossible in a world with multiple GPU vendors, and even things like AMD replacing its machine code format in its own product line.
Also, we don't _want_ to do this direct hardware coding so much as we _have to_ to make sure the games we put out in 2012 looked better than our competitors' games in 2008 despite using the exact same hardware. We have to push micro-optimizations and tricks to improve performance because the hardware is locked in the stone age.
Consoles can be faster than an equivalent PC because we have no real OS and can micro-optimize in ways that PC/phone developers can't. This is not a good thing, for developers or users. Users want diversity and competition, and developers want easy APIs that make development cheap and efficient. Consoles do neither.
Meanwhile, real PCs can massively outperform consoles because for all the bloat, the hardware is massively more powerful, partly because many companies can compete to make the best hardware and users can and do upgrade at will. So mmarq's argument is just silly. Hardware does bring massive performance improvements, and software trickery is about making the most of limited hardware.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 21, 2013 9:18 UTC (Thu) by khim (subscriber, #9252) [Link] The only games which "make way more use of the hardware in consoles" are exclusives like Uncharted where you really can throw away all these HSA discussions and code to bare metal. Most cross-platform games (and very few games are exclusives nowadays) are better in XBox360 because it's hardware is closer to PC and usually you don't need 10x more powerful PC to beat the console game in quality (2x is more then enough) which implies that these "micro-optimizations and tricks" are not all that popular. They are used in some "computationally heavy" places (because otherwise game will just not work on console) but most of the time it's just plain simple reduction in quality (number and size of textures, etc) till you have the required FPS. And of course there are no HSA or "bare metal" access: it's OpenGL or Direct3D on consoles, too.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 5:51 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
oh, you mean all those devices that I cannot run a current version of the OS on because the vendor drivers don't work, or are not made available.
These vendors are examples of where things are not working, not examples that we should be following.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 8:23 UTC (Mon) by mmarq (guest, #2332) [Link]
Don't worry life always finds a way...
I don't know what is the politics of HSA what they decided... but i think they have serious problems... and drivers is not the only concern...
They must have an *Open OS*... for their *Open Standards* -> they are open, all GPL compatible(Double licensing may be used, but also BSD style is used and not only LLVM)
IBM is not in it(doubt ever will, its not its targets, unless they pass to use GPGPUs and stuff for computation, which also doesn't make much sense for them )...
Apple though based on ARM will not go for it, open their OS, even if they join...
Microsoft the same or worst...
BSD are too lacking for "devices & stuff" and getting obsoleted by Linux
*\\It remains only Linux//*
More than drivers, i think it would be wise to see if they could build ALL of an OS, if not almost all of a Distro, using their very comprehensive compiler toolchain... and which could very well be ***much more ARMv8 64 bit centric than x86***.
GCC is in a paradox crosswords as reported by Pharonix, Intel ICC is too CPU centric and will remain as foreseeable, even if they open source it, and worst doubt they will ever target ARM...
Since they set Most of Industry, including the leaders of the client side of computing, ARM Samsung & CE, to build a top nosh compiler toolchain designed specifically for this... one natural candidate might be Android...
But i can't see if Google is up to it, they may decide to go a little like Apple/M$ and target primely their stuff... its Linux based, its some proprietary to, and they might NOT be to take the BURDEN...
HSA is big, they can do it...
So what only remains is fork the Linux kernel itself... I SEE NOTHING HOSTILE IN THIS... IT WOULD BE EXACTLY LIKE *ANOTHER* DISTRO BUT ONE THAT WOULD GO MUCH DEEPER THAN THE USUAL, WITH THE APPLYING PATCHES NOT APPROVED/WAITING MAINLINE... ALL DISTROS DO IT MORE OR LESS...
In the end LF and Linux can also benefit from it, it will be like another branch, and it already maintains several, so another one maintained elsewhere doesn't hurt... and build with HSA CC, and with its driver interfaces, which could be DKMS++ based and used also by many other distros ... and who knows if not useful in the future, who knows about GCC -> can it evolve ?
I'm afraid with the politics not all is rosy, or if anyone can ask more than the remarkable work that has been done so far by the LF, but when the HSA compiler collection is finished, they must find a real USEFUL solution and they would want to really test it with an OS... they will have no other choice, but to find that solution -> you simple don't waste a lot of time expertize and money building something and then not use it...
Don't know politics, but i see it like an addition... not something bad... distros, for everyone that flanked along this years, 2 more seemed to have popped out in its place, many with many peculiarities, everyone trying to be different... what would be the danger of having another one ??...
It would be not an attempt to take over... matter of fact i think exactly like any other distro, they would link LF attentively and try to participate, only they could go very different in a lot of changes...
The GPL is not really suitable for "exclusivity or my way or the highway kind of approaches", who ever thought of that must be desperate or in deep delusion...
In the end i don't know the decisions... but also i don't see many other options...
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 9:12 UTC (Mon) by mmarq (guest, #2332) [Link]
"exclusivity or my way or the highway kind of approaches"
I mean it is used, and could be used extensively... but doubt it will ever produce any political results for any particular agenda or impediments to others... besides exactly a cave-in choice from the producer... and doubt any "forcing" effect in any sizable receiver
That could have worked more or less for the type of distros he had so far... and even so not all... and none in strict patch software feature choice..
Linux as been protected by the inherent complexity from fork pressure and be able to dictate...
Android breached that somehow... in the future it can be much worst... and its not only about HSA...
So i find this comments about "my way" kind of vision and we should pursue this and not that very amusing... probably not conscious kernel devs, if one at all... and the case is you should pursue what you think best, but abstain from such comments.. lol...
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 19, 2013 19:25 UTC (Tue) by khim (subscriber, #9252) [Link] Samsung, ARM, TI, Sony, AMD, Imagination, LG, STMicro, ST Ericsson etc etc etc... are academic guys without real hardware ? Nope. These are companies which cough up few millions here and there to make sure they'll be in the loop if this idea will actually reach some usable stage. Does not mean anything beyond that at this point. Remember the last "next big revolution" (called EPIC if you forgot)? It also had an impressive names: Intel, HP, IBM (which wanted to use the opportunity to unify UNIX), etc. They had really awesome slides! And really cool story. I'd say EPIC story. In the end we've got an epic fail, nothing more. I'm not saying they'll fail but the fact remains: number of supporters and sizes of such supporters don't automatically equal success.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 19, 2013 23:41 UTC (Tue) by mmarq (guest, #2332) [Link]
You are all mix up... lol...
HSA is NOT about a CPU u-arch... at best its about a macro-arch concept, one which is highly flexible... but most of all is about a "PROGRAMMING PARADIGM"...
So according to the names involved, because *they use it*... HSA fits nicely with ARM, x86, MIPS and PowerPC(thought Sony changed, i think its used on some other stuff by some) on the CPU side... TI, ARM, MIPS etc DSPs... all the analogue stuff of ST-Ericsson and MIPS and ARM etc... and its open to many more!...
So catching a few millions here and there, all those guys make many times the size of Intel as example (5x perhaps !?)... Only Samsung itself is bigger than Intel... no matter how much many dogmatic sectarian POV wants to twisted it, it is truth...
Also no matter how hard it takes to many... can be debatable, but highly dependent on how and what you count... from Smarth/Superphones to desktop, by *unit accounts*, the CPU u-arch leader of the client side of computing is ARM not x86... an nothing will prevent with ARMv8 64bit, the spill in force into the traditional Mobile (laptop) side of computing and even some desktop( in this earlier steps)... not even if Intel goes thermonuclear (its too late now).
I think Google saw the light... "renderscript" smells HSA allover some how, and staying more ARM centered and not MIPS or x86... Android will be the only real fight to Microsoft in the client side... though Apple will remain very big, as big as Intel, i think they are "too closed" to ever be the clear dominant force on the client side.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 20:11 UTC (Thu) by mmarq (guest, #2332) [Link]
Forgot!
I don't know why you gentlemn fear or oppose some of this kernel inclusions. It could be modular a compile option, and import little additional maintenance burden... or the opposite, if with those it imports also additional savyy developers.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 15, 2013 18:18 UTC (Fri) by nix (subscriber, #2304) [Link]
Uh, no. That's the worst of all worlds, since it implies maintaining two separate bodies of code, in-the-kernel and out-of-the-kernel, where only one of those is likely to get much testing. There's a reason userspace modesetting is generally torn out of X.org video drivers once that driver's KMS support is stable: it replicates the code in KMS in a less capable and harder-to-maintain fashion.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 18, 2013 4:57 UTC (Mon) by mmarq (guest, #2332) [Link]
If you talk DKMS don't worry... it only have to provide decent support and not even DKMS itself... then any distro on earth will DKMS you in the face every time... sorry, ***but is exactly what they do already***... you don't use Linux or you build from scratch... the possible conclusion...
ummm... stange agendas !... DKMS is exactly a good idea for the overburden complains. Don't know why Kernel devs must maintain every thing about drivers... lots of bickering yet it already imports firmware blobs, when there could be a proper interface (discussed/DESIGNED case to case of family, and parts of DKMS changed accordingly)...
Then the more generic stuff goes kernel maintenance... APIs don't change that often(matter of fact quite unoften for many tastes)...come on you know its truth!... and the burden to support every single variant of a hardware family goes to the vendors... like half half...
Free drivers could co-exist with proprietary,*** which they already do***, so nothing new and nobody got hurt, and if properly designed, more things could be open and free drivers use a lot of the same stuff of the proprietary ones without the need to be a mess of half finished things and proprietary drivers that break at every 2 versions...
And if security bitching is a concern, why not direct the program caging and sandboxing to the DKMS side and really support it, instead of shoving it to the unsuspected user, that by no dreams will ever be using NSA grade...
About KMS it makes good sense in userspace... the reason is that consoles will also want to use the possibilities, kernel interfaces directly, so its sensible...
The other arguing, maintenance, stuff... issues... preferences... tastes...particular visions.... semantics for pointless bickering.
10 years and little change to many things.. oh! well, progress is not painless... action counter-reaction applies here to... (damn politics!)
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 25, 2013 17:02 UTC (Mon) by nix (subscriber, #2304) [Link] The other arguing, maintenance, stuff... issues... preferences... tastes...particular visions.... semantics for pointless bickering.So... not having to implement everything twice is 'semantics for pointless bickering'? Ooh-kay.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 5:51 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
> Aside from that having been historically true, back when hardware was MUCH simpler, why should that be a particularly important distinction today?
because kernel developers care about backwards and forwards compatibility, userspace developers tend not to.
As an example of where 'put it in userspace' is an ongoing disaster, look at camera 'drivers' for android devices. This is done in userspace, but there are a lot of not-so-old android devices that cannot use the camera on newer versions because the closed source userspace drivers don't work with newer versions of android.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 7:45 UTC (Thu) by mmarq (guest, #2332) [Link]
"" while there were significant advantages in maintainability, error recovery, and general robustness to having that driver code be in user space. ""
Naa!.. that was because your kernel (in kernel space) wasn't nearly as good as Linux... lol
Seriously, i think of a parallel, Virtual Machines... ummm... why don't run the all shebang in userspace ??
Almost impossible for many tasks & functions, right ??
A driver for a full blown GPGPU is the same. Things have changed a lot from earlier models... things can change dramatically in near future, you'll see... research like you mention is not only curious, it will be mandatory to do again.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 15:11 UTC (Thu) by brouhaha (guest, #1698) [Link] Naa!.. that was because your kernel (in kernel space) wasn't nearly as good as Linux...Linux (in kernel space) wasn't nearly as good as Linux? No, I don't think that was the reason.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 19:55 UTC (Thu) by mmarq (guest, #2332) [Link]
without wanting to molest feelings, but Linux how long ago ?
If it was as good as is now, why the fuss with all this development... make it worst ?
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 17:21 UTC (Thu) by raven667 (subscriber, #5198) [Link]
> we put nearly everything that touched the hardware in user space, other than (as I suggested for graphics drivers) small kernel-space drivers to reflect interrupts to user space
Then you are actually agreeing with the existing KMS design since that is pretty much how it is implemented. The small amount of hardware management and IO needed to get data to/from the GPU is implemented as a kernel driver and the bulk of the complexity is in a userspace library (libEGL/libGL). The area where there may be confusion is that the minimum viable complexity of the kernel driver may be quite a bit higher than hardware you were interfacing with in your example, the GPU is practically a whole other machine with its own CPU, RAM, I/O but also sharing with the host machine. There's no way to manage that without some level of cooperation between the GPU and the CPU kernel.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 7:28 UTC (Thu) by mmarq (guest, #2332) [Link]
GREAT! run the linux kernel in userspace also... leave only there XEN or better called Nemesis...
That is the Exokernel approach... that happen can work very well and faster for many "things"... only i think Linus will let the dogs on anyone that dares to suggest that... lol...
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 14:09 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
So you're describing YetAnotherMicrokernelOS. Fine, go on and start writing it.
But please, leave Linux alone - we really would like it to succeed, not crash and burn like all other microkernel OSes.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 19:50 UTC (Thu) by mmarq (guest, #2332) [Link]
No... i was being a little sarcastic... lol
Point was, since the pointless bickering tends to be black or white... almost everything on one side or the other... i was just describing the option for a micro-micro-micro kernel (exo) (choose black or white as suits your fancy lol).
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 6:09 UTC (Thu) by mjg59 (subscriber, #23239) [Link]
GPUs can do DMA. Allowing userspace to submit arbitrary commands to GPUs means that userspace can do arbitrary DMA, and as such means that any application with access to your GPU can do anything it wants to. As such, you need command validation in the kernel in order to prevent userspace from being able to cross security boundaries. The kernel needs DRM support to validate the commands that userspace submits.
Modesetting. The kernel needs to be able to reprogram a mode over suspend/resume. That means that the kernel needs to be able to program the mode that userspace requested. You need modesetting support in kernel. At that point, allowing userspace to program a mode that doesn't match the kernel's expectations is dangerous. The kernel needs to be able to set modes.
There's plenty that the kernel does that can't be handled in userspace, simply because (a) we don't trust userspace, and (b) userspace doesn't run during suspend/resume. You could write an operating system that did all of that in userspace, but it'd be shit and incapable of meeting modern expectations of OS security. The only reason to push hardware management out of your kernel is because you're producing a microkernel, and Linux isn't.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 6:25 UTC (Thu) by brouhaha (guest, #1698) [Link] GPUs can do DMA. Allowing userspace to submit arbitrary commands to GPUs means that userspace can do arbitrary DMA, and as such means that any application with access to your GPU can do anything it wants to.Which is why you wouldn't let just any userspace application access the GPU directly. I don't let just any userspace application scribble on /dev/sda either. I'll accept your argument for modesetting for suspend/resume. There's plenty that the kernel does that can't be handled in userspace, simply because (a) we don't trust userspace,That's a bizarre argument. Having code in user space doesn't magically make it less trustworthy than the same code would be in kernel space. Whether particular code should be trusted is a matter of policy, and there are various mechanism for policy enforcement. The only reason to push hardware management out of your kernel is because you're producing a microkernel,Assumes facts not in evidence. At the very large router company, we pushed hardware management out of the kernel for many reasons, none of which had anything whatsoever to do with whether the kernel we were running was monolithic or a microkernel. I could certainly accept that *A* reason to push hardware management out of your kernel is because you're producing a microkernel. It's a big jump from *A* to *The only*.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 6:33 UTC (Thu) by mjg59 (subscriber, #23239) [Link]
"Which is why you wouldn't let just any userspace application access the GPU directly."
Yes! You shouldn't let any userspace application access the GPU directly, because these days we have an expectation that userspace shouldn't be able to compromise the kernel. The alternative is to have signed userspace, and that's not an acceptable option. So, unless you're producing a microkernel which has a separation between drivers and the rest of userspace (which Linux doesn't have), the correct line to draw is the one where only the kernel gets to drive hardware that can overwrite the rest of the OS.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 14, 2013 17:27 UTC (Thu) by raven667 (subscriber, #5198) [Link]
> That's a bizarre argument. Having code in user space doesn't magically make it less trustworthy than the same code would be in kernel space. Whether particular code should be trusted is a matter of policy, and there are various mechanism for policy enforcement.
Sure it does, the kernel has no control over what is run in user space and can make no assumptions about it being the "same code" or some expected implementation, it can have zero trust that the data given to it from userspace isn't bogus and absolutely must check everything. This is different than a closed environment like an appliance where one entity has control of both userspace and kernelspace, maybe restrictions could be relaxed in that case.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 11, 2013 17:31 UTC (Mon) by daniel (subscriber, #3181) [Link] Maybe we can move Firefox and LibreOffice into the kernel next.I do not need to reboot Linux often, but when I do, the reason is usually DBus. Often because DBus has the CPU pegged at 100%, an effective DoS. Killing DBus usually leaves the system in an unusable state from which there is no obvious recovery. I say, if there is to be a world-eating message bus in the kernel, design it properly. DBus is not ready for prime time, far from it.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 11, 2013 17:56 UTC (Mon) by hp (subscriber, #5220) [Link]
"I found a bug I didn't root cause. Clearly it is a fundamental design flaw!"
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 11, 2013 19:27 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
Can you post details? This looks like a very unusual bug - we're using DBUS on tons of devices and quite a lot of them have uptime measured in _years_ (they are not connected to the Internet and communicate through RS-232/485, so no need to worry about most security bugs).
My personal gripe with the DBUS is its poor handling of reconnections.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 11, 2013 20:54 UTC (Mon) by hp (subscriber, #5220) [Link]
You can find some hashing-out of the reconnect thing here:
http://lists.freedesktop.org/archives/dbus/2005-March/002...
I don't think the problem has anything to do with dbus-daemon; you can restart it fine.
The problem is that apps don't handle the restart... and that for them to handle it would require them to write quite a lot of complex code that would rarely be tested.
Say hypothetically that someone wrote all that code, and then religiously lobbied app developers to keep writing it in new apps, and kept testing it and fixing bugs...
Even given this hypothetical work, personally I would never trust that at a given point, I could trust all apps to have that codepath working. So I would just reboot anyway.
The difference between dbus and other daemons here is not that dbus somehow forbids restart. It's that dbus has persistent and stateful connections (and that's core and essential to the purpose of dbus).
Restarting dbus is like saying you want to restart the X server without killing any X apps. It's the same technical challenge as that. Namely, all apps would have to track and be able to restore all the state kept by the server.
Rule of thumb with dbus: what does X protocol do? dbus usually does the same thing and has the same pros and cons.
We have looked some at a client library design that makes it easier to handle daemon restart - basically a library where you provide a cascade of callbacks ("connect to bus handler", "service is now owned handler", etc.) and those callbacks could be re-run on bus re-connect. However, it is a lot harder for app developers to understand this kind of API, and in any case, existing apps aren't doing it this way.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 11, 2013 23:04 UTC (Mon) by daniel (subscriber, #3181) [Link]
So, if I may humbly misquote the content of your post: we are on this day, presented with a rare and wonderful opportunity to introduce a whole new class of kernel bugs, which so far have been mainly relegated to userspace and mainly manifested by the likes of Gnome and KDE apps. By no means should we miss this wonderful opportunity to grab this shiny new tool by the trigger and blow our collective legs off.
Of course, we could always consider waiting for DBus to stop causing user space issues before welcoming it into kernel with open arms, hearts and minds. I'm trying to avoid using the word "gaping" here...
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 11, 2013 23:10 UTC (Mon) by hp (subscriber, #5220) [Link]
I have no idea what you're trying to say.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 12, 2013 0:04 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
That's the problem - the connection between the dbusd and clients has too much state. It could have been made stateless, or at least with the state confined to the client-side library.
It's not really possible with the current DBUS protocol. That's another reason why naively networked DBUS is not such a good idea. However, layering DBUS on top of something like ZeroMQ could be interesting.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 12, 2013 1:00 UTC (Tue) by hp (subscriber, #5220) [Link]
It's meaningless to say "too much state" in the abstract without understanding what the state is and why it exists.
dbus was widely adopted because it solved certain problems that were previously unsolved, by making different tradeoffs vs previous solutions.
Anybody can show up and say "oh that tradeoff has this downside." That's why it's called a tradeoff.
Anyone on the Internet could prove me wrong by showing the code which has the pros without the cons. That's the beauty. Everyone would jump to use a best of all worlds solution like that. Meanwhile, people are using a solution that exists.
In my view, the client libs could be designed to better support reconnection but ultimately the app has to handle the case. Neither the daemon nor the protocol are the source of the "restart problem." The problem is that handling restart in N different codebases, with none of them ever buggy, is not practical. It isn't impossible, but nobody who has actually written code, to date, has decided the cost:benefit ratio holds up and proceeded to tackle this.
Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel Posted Feb 12, 2013 2:22 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
Hey! I'm not saying that DBUS is somehow completely unacceptable and bad.
Yes, it works pretty well. But it does have shortcomings that could have been avoided by a more careful design. You can make a "reconnectable" messaging protocol pretty easy, it's not rocket surgery - by storing the current state of the server's subscriptions in the durable storage, for example. Or by introducing an explicit "reconnection" phase.
Slightly off topic (Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel) Posted Feb 12, 2013 10:24 UTC (Tue) by ortalo (subscriber, #4654) [Link]
"rocket surgery": sounds like a nice new discipline to me, isn't it?
I am left wondering if it involves doing surgery in space or repairing rocket engines? (Hopefully not both...)
Slightly off topic (Kroah-Hartman: AF_BUS, D-Bus, and the Linux kernel) Posted Feb 12, 2013 14:01 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
Rocket surgery usually involves doing neurosurgery. With ROCKETS!
|
|||||||||||||