Weekly Edition
Return to the Development page
| |||||||||||||
Graesslin: Driver dilemma in KDE workspaces 4.5
Martin Graesslin looks
at problems with the interaction between KWin and some graphics drivers.
"Now that I have explained all our checks we did to ensure a smooth
user experience, I want to explain how it could happen that there are
regressions in 4.5. In 4.5 we introduced two new features which require
OpenGL Shaders: the blur effect and the lanczos filter. Both are not hard
requirements. Blur effect can easily be turned off by disabling the effect
and the lanczos filter is controlled by the general effect level settings
which is also used for Plasma and Oxygen animations. Both new features
check for the required extensions and get only activated iff the driver
claims support for it. So everything should be fine, shouldn't it?
Apparently not when it comes to the free graphics drivers (please note and
remember: we do not see such problems with the proprietary NVIDIA
driver!)." (Thanks to Jos Poortvliet)
(Log in to post comments)
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 20:36 UTC (Mon) by drag (subscriber, #31333) [Link]
Yeah....
The same problems that are faced with Audio and Wifi drivers are now being faced by video drivers:
Inconsistency caused by drivers lying about features they support, lack of testing, lack of users using newer/unusual features, etc etc.
This has been going on for a long long long time and because we are dealing with open source applications are expected to be modified and tested to deal with these inconsistencies. It's not policy: it's just a effect caused by the historical suckiness of Linux specifically and hardware in general.
But because of the open source nature of Linux the problems can be fixed:
That was the mistake of the Kwin/Gnome-Shell/Compiz and other systems like this. They need to remove all checks, all hacks, all workarounds and simply take drivers at face value. Trust the drivers to be able to do the things they say they can do. If they say they can do GLSL then use GLSL. If they can do OpenGL 2 then use OpenGL 2. That's all. If it does not work then tell the driver developers so they can fix it.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 1:40 UTC (Tue) by RCL (guest, #63264) [Link]
It's actually impossible even on well-tested platforms with good support (read: Windows). Even with such a standardized and clean API as DirectX (3D, Compute) is.
Linux sucked for anything graphics related since forever and NVidia has been a lone saviour for a number of years. That problem isn't new. KDE can't "fix them in the drivers" because nobody - not even major commercial Linux players like Red Hat - was able to do that either. Hell, even Intel can't, despite having its own drivers for its own hardware.
That's all is pretty sad, actually. With the appearance of GPGPU more than 4 years ago, the world is slowly moving to some hybrid computing model where main CPU cores are used only for task-parallel programming, which is a minor challenge, while data-parallel stuff - the most compute-intensive one - is being done on simpler, but more numerous GPU-like cores.
Good news is that the future indeed belongs to multicore systems. Bad news is that it won't be symmetric, but rather asymmetric multiprocessors with most of performance-critical code being written in "virtual" instruction set like NVIDIA's PTX which heavily relies on drivers to translate it to hardware-specific representation.
GPU drivers nowadays aren't just for graphics. We're abstracting away the processors (which is a good thing), but unlike Java etc we avoid losing the speed that much. Unfortunately we are introducing a layer of drivers which will soon be needed to run even the most basic software. And there's no open source solution for that, not even on a horizon - seeing how open source drivers are struggling to support the functionality of pre-GPGPU era...
Actually the problem becomes easier, not harder Posted Sep 7, 2010 6:12 UTC (Tue) by khim (subscriber, #9252) [Link] GPGPU does not introduce new hardware, it's just a way to use the same hardware for different tasks. This means when free drivers will be usable for 3D work they'll be easily adopted to GPGPU work. Advent of GPGPU era also means differences in architecture will be less and less important: strange limitation of different 3D chips which affect 3D drivers today will be just removed since GPGPU requires significantly more uniform architecture to be usable. Hardware may be totally different (like Athlon and Pentium4) but software should see something quite similar - or else the whole thing will not work.
Actually the problem becomes easier, not harder Posted Sep 7, 2010 6:45 UTC (Tue) by drag (subscriber, #31333) [Link]
My understanding is that nowadays there is very little '3d' done in hardware. No 'opengl state machine' or anything like that. It's all software rendering with just some of the software designed to run on the GPU while other parts on the CPU.
This is why it's so important to get working open source drivers. The GPU is now part of the normal computer architecture, just like the math co-processor from the 386/486 days. Just exponentially more complicated.. Imagine having to install 'drivers' to use your central CPU! Depending on proprietary drivers to create a abstraction between you and your hardware in this manner is insane if you want to maintain that you run a open source system.
It will not be long until the GPU will be integrated as cores on your central processor. It's not a matter of 'if', it's only a matter of when. Moore's law is going to continue to go into effect and it's just 'duh' that those transistors are going to have to be used for more then just another general purpose core if we want to continue to see performance increases and increases in efficiency.
Shuffling information over PCI-Express bus is going to start to get extremely expensive no matter how much bandwidth you have... it's latency that is the killer.
Actually the problem becomes easier, not harder Posted Sep 7, 2010 9:18 UTC (Tue) by nix (subscriber, #2304) [Link]
The fact that you need to install drivers on your GPU, but not your CPU, has nothing to do with the complexity of the GPU per se. It's got to do with the fact that the kernel (and drivers) are running on the CPU, so something on the CPU has to program the GPU, maintain a shared memory management layer, and tell the GPU what to do. And that something is the driver.
Actually the problem becomes easier, not harder Posted Sep 7, 2010 9:50 UTC (Tue) by RCL (guest, #63264) [Link]
I like your comparison of GPUs to FPUs... It's indeed the case :)
By the way, just to illustrate the importance of GPGPU in traditional areas: http://gpgpu.org/2010/09/06/packetshader-software-router
Actually the problem becomes easier, not harder Posted Sep 7, 2010 11:16 UTC (Tue) by Trelane (subscriber, #56877) [Link] GPGPU does not introduce new hardware, it's just a way to use the same hardware for different tasks.Jein. See also, NVidia Tesla. Perhaps more correct would be that it doesn't introduce new chips; TMK Tesla cards are just specialized hosts for standard higher-end NVidia chips. strange limitation of different 3D chips which affect 3D drivers today will be just removed since GPGPU requires significantly more uniform architecture to be usableDepends on what you're referring to. E.g. GPUs have a pretty strange memory hierarchy, and I don't see it going away. GPGPU programmers just work around it for the most part as far as I can tell.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 7:16 UTC (Tue) by drag (subscriber, #31333) [Link]
> Linux sucked for anything graphics related since forever and NVidia has been a lone saviour for a number of years. That problem isn't new. KDE can't "fix them in the drivers" because nobody - not even major commercial Linux players like Red Hat - was able to do that either. Hell, even Intel can't, despite having its own drivers for its own hardware.
Linux has made huge strides and will continue to do so. There is no choice. You cannot depend on Nvidia for making a working system that will be competitive with Windows or other operating systems. Nvidia does not give a shit one way or another about Linux. _They_just_don't_
Unless your a major OEM or are willing to spend huge amounts of money on high-end professional graphics for Linux Nvidia does not care about you. They don't care about your desktop, your kernel, your needs. They don't care about making it easier, more secure, more stable, more efficient or anything remotely like that. The only reason we have the drivers we have is because Nvidia's consumer hardware is so similar to their professional hardware and they have the ability to make a shim into the kernel to shove parts of their XP drivers into it.
Anyways: GPU's have been very 'General Purpose' for a very long time now. Ever since they started calling them 'GPU's, basically. Having to build compilers to build and execute GLSL code on them is proof of this. It was a huge failure for XFree86 and other folks to not recognize this and they delayed modernizing the driver architecture for nearly a decade. ATI and Nvidia proprietary drivers made the transition to the newer architectures years ago and since then the move to 'GPGPU' has been a evolutionary one.
If Linux cannot provide the ability to perform well on modern hardware without shoveling well over 50MB of Windows driver code into the system then what exactly is the point of having Linux, then? Just so you can have a good command line or something? I can easy show you how to have a POSIX environment on Windows 7 if that is all you really care about. (One of the nice things about NT Kernel is that it has the ability to support multiple user space APIs.. the win32 stuff is just one of many it supports.)
How can Linux move forward and be able to claim that it supports hardware with 90% of peoples computers will not have the ability to support modern APIs? Only a small minority of people actually do run modern Nvidia hardware that is supported by Nvidia drivers... 70-80% run Intel hardware and ATI is still popular.
That is what is so sad about the developer complaining he did not have the hardware to test open source drivers. Only a small fraction of his potential audience has the ability to run proprietary Nvidia drivers. Even if all he had was nvidia hardware it is still more then likely supported by open source drivers.
How can you have modern applications and modern graphics when the software support for it in Linux is that spotty? Why do you think that game developers swore off Linux as a nightmare to program for?
We now have working Gallium3D drivers. We have working open source drivers on 3 of the top most common hardware families out there which will cover well over 95% of the x86 hardware out there.
Things have improved hugely and when Intel finally gets off their asses realizes that they need to use the same driver architecture that ATI and Nvidia open source developers are using then Linux will be able to provide the same level of consistency that things like DirectX and Windows provide. It will be years before it reaches the same level of modernization and performance, but that is much less important then having something that is reliable and works consistently across the board.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 14:16 UTC (Tue) by fandingo (subscriber, #67019) [Link]
Simply not true. How can you explain VDPAU? There isn't much commercial application for it. VDPAU is completely a consumer technology.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 10:16 UTC (Tue) by NAR (subscriber, #1313) [Link] Never try to work around hardware and driver bugs at higher levels if you can help it.Wasn't this exactly what the pulseaudio developers did? It wasn't that nice from the user's perspective...
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 12:23 UTC (Tue) by nix (subscriber, #2304) [Link]
Unfortunately the hardware bug that modern sound cards can't be bothered to do mixing doesn't seem to be going away.
(In any case, it is routine for drivers to work around hardware errata. What else are they supposed to do? Say "no, get another motherboard" whenever a new erratum comes out?)
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 13:45 UTC (Tue) by rvfh (subscriber, #31018) [Link]
> (In any case, it is routine for drivers to work around hardware errata. What else are they supposed to do? Say "no, get another motherboard" whenever a new erratum comes out?)
Drivers yes, applications no. They should not. It's the job of the driver _only_ to hide the hardware specificities and bugs.
But I'm sure that's what you actually meant :-)
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 17:46 UTC (Tue) by alankila (subscriber, #47141) [Link]
Mixing with volume changes is relatively tricky to do well, as the audio must be dithered for acceptable output quality. Also user may want to add some sound processing in the output path, and that is actually a task best left to the host CPU, because such needs can not necessarily be anticipated at hardware/driver development stage. Finally, audio is cheap to process with the host CPU, no dedicated hardware is required for it. In short, I believe hardware mixing is less useful than it sounds.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 14:02 UTC (Tue) by zdzichu (subscriber, #17118) [Link]
The other way around. But the situation is very similar. We have piece of software that begin to use parts of API never really tested before. And bugs are uncovered.
With PulseAudio, Lennart took some work to report and fix kernel bugs in audio drivers, and to describe what is needed to ALSA developers. This way we have mostly working audio drivers now.
Similarly, Dan Williams fixed bunch of wifi drivers when NetworkManager uncovered their sad state. There were some workaround implemented in NM in the meantime, but at the end of the day drivers got fixed.
This need to happen again. Bugs got uncovered, need to be fixed. Consider KWin a testcase, real world one.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 15:30 UTC (Tue) by NAR (subscriber, #1313) [Link]
However there is a slight difference: WiFi doesn't work? I plug in the UTP cable. Audio doesn't work? I turn on the radio instead of listening to MP3. But I really can't use the computer if the screen is blank or the window manager crashes all the time... Care must be taken when to turn on this feature by default.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 15:48 UTC (Tue) by lkundrak (subscriber, #43452) [Link]
Do you also listen to radio instead of VOIP calls? Or do you usually find UTP cables connected to the internet when at the places with hotspots?
Come on, broken wireless and broken sound is no less severe than broken KWin.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 1:00 UTC (Wed) by cesarb (subscriber, #6266) [Link]
I think his point is that, if your video output is not working, you cannot use your wireless or your sound at all, even if both are working perfectly.
Few people use serial for output to the user these days, and we are talking about desktops so no headless like servers.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 9:55 UTC (Wed) by niner (subscriber, #26151) [Link]
You're completely exagerrating. If your drivers do not support kwin effects you still can simply turn them off. Or use another window manager. Or use plain text console. It's simply not like you cannot use your computer without kwin's blur effects
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 12:18 UTC (Wed) by tialaramex (subscriber, #21167) [Link]
If your drivers simply "do not support" the effects, sure. In fact in this case KWin will disable the effects itself.
But there's a good chance (and the article was about this) that the driver claims to support the effects, but instead it corrupts the graphics, or renders everything transparent, or hangs the machine solid.
Imagine if, instead of the situation where Linux merely doesn't support your WiFi chipset, it claimed to support the chipset but every few hours downloads stalled for no reason and you had to reboot to fix it. Or worse, after anywhere from 5 minutes to a day the machine just locks up completely.
Such bugs were very rare with WiFi, and rapidly fixed, but with graphics they have been omnipresent since the first integrated 3D graphics chipsets in the late 1990s.
Running the plain text console on a modern desktop or laptop is not a legitimate solution, any more than "WiFi works for me so long as I never run anything more than a single telnet session".
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 14:49 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
"Imagine if, instead of the situation where Linux merely doesn't support your WiFi chipset, it claimed to support the chipset but every few hours downloads stalled for no reason and you had to reboot to fix it. Or worse, after anywhere from 5 minutes to a day the machine just locks up completely."
Sounds pretty close to reality for me. Our wifi support is certainly better than it has been at most times in the past, but I still occasionally end up needing to unload and reload iwlagn to get it to behave itself.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 15:05 UTC (Wed) by Trelane (subscriber, #56877) [Link]
what is iwlagn's problem anyway? It seems like the buggiest driver anymore.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 15:56 UTC (Tue) by SEJeff (subscriber, #51588) [Link]
The same could be said about NetworkManager, except now the upstream is mostly awesome.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 21:43 UTC (Tue) by dcbw (guest, #50562) [Link]
It's only awesome because NM pushed the envelope with what was expected of drivers (like KWin), I spent years pushing fixes upstream into the drivers (unlike KWin), and evangelizing the kinds of things NM and users wanted from drivers to upstream developers (apparently unlike KWin). After a while some great people came along upstream and really pushed things forward so that we have the situation today where wifi is pretty much "done" for most people.
Perhaps the KWin people should redirect some energy into patches and evangelizing.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 20:54 UTC (Mon) by biehl (subscriber, #14636) [Link]
There's more discussion in the phoronix forum
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 21:22 UTC (Mon) by Sho (subscriber, #8956) [Link]
Pretty hard to sift through, 75% of it is fairly uninformed trolling.
A frequent point made in that thread is that KDE should try harder to work only within the limits of the feature set consistently supported by open source drivers, as reaching beyond supposedly plays into the hands of the proprietary driver makers (i.e. nVidia) by making our desktops dependent on the larger feature set offered by them.
I don't think it's quite that simple, personally. First of, the code wishing to make use of these features is itself implementing optional features, and the trouble right now is not in relying on these driver capabilites but in being able to reliably tell whether the driver supports them.
Even when taking a few steps back from the specific problems, it's noteworthy that KDE 4.x actually does not depend in OpenGL in any way - compositing itself is entirely optional to begin with, but the compositing manager also has a 2D-only XRender fallback that doesn't require OpenGL acceleration. This is different e.g. from Gnome 3's new Gnome Shell, which has a hard dependency on OpenGL (afaik, please correct me if I'm wrong here).
But then I also think that the desktops *should* push the envelope and try to exercise driver functionality that's in active development (especially when the proprietary competition has often supported it for multiple years). It's useful feedback for driver developers what to spend their time on, and I think in some sense it also helps recruit new driver developers: whether something is interesting to hack on is to a significant degree related to whether it's in demand. Most hackers would rather spend their time on code that is going to see wide use - it's just more satisfying, even aside from potential fame and fortune.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 21:41 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]
It is great to push the edge and try to do more but the testing should be separate from actual production releases. Kwin developers could have called for a public round of testing from end users so that results from a diverse set of hardware and drivers can be tested on before assuming that the strategy that they have planned will work. Regardless of whether it is the fault of Xorg drivers (agreed that it is and there is a resource shortage problem there), end users updating to a newer version of KDE will see a large numbers of regressions and KDE project will be responsible for that. You cannot explain away that problem. It requires a lot more collaboration between Xorg developer and KDE developers.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 21:48 UTC (Mon) by Sho (subscriber, #8956) [Link]
> Kwin developers could have called for a public round of testing from end users so that results from a diverse set of hardware and drivers can be tested on before assuming that the strategy that they have planned will work.
There were two betas and three release candidates for KDE SC 4.5.0. I don't really know why that wasn't sufficient. From what I've read the problem seems to be that there's a fairly high bug noise ceiling due to driver flakyness that makes it hard to tell whether a report describes a one-off problem or if there's in fact a single problem that affects many people. So it becomes a question of how to produce better data to improve the metrics, I guess.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 21:53 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]
You answered your own question. Call for testing should be focused and explicit guidance needs to be given on how to get started and report feedback. I am biased but I quite like Fedora test days and while it is not perfect does usually help find quite a number of issues and resolve them fairly quickly. Recent example
http://lwn.net/Articles/403677/
Fedora does explicit test days for Xorg drivers and that could very well be adopted to solve some problems seen by KDE.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 1:21 UTC (Tue) by tialaramex (subscriber, #21167) [Link]
The outcome of those tests is very consistent:
For most people it doesn't work. They fill out rows of N/As in the test charts, or they leave the test channel after wasting hours trying to follow broken instructions written by overworked QA staff who are expected to make up for the disinterest of the "maintainers" of the driver.
And I think I finally figured it all out, the problem is that the Wise Men involved perpetually think it will be finished tomorrow, so no need for scaffolding. If I am building something as fragile and dangerous as KMS, every other line is debug. At each step I am trying to figure out what's wrong now, and when something is wrong, I shout it from the rooftops, in the hope that someone will hear before the world ends. But my approach is defeatist, it asserts that we are trying to find the problem, which means there is a problem, and nobody wants to hear that.
Try it, report a critical bug in Rawhide. It locks solid on your machine. Yep, looks like the KMS driver. And then you'll get radio silence. They have no mechanism to debug the hang, even though such hangs are an ever present menace of low-level graphics chipset tinkering. They may tell you to keep trying new versions, maybe it'll fix itself. That's all they have.
I remember when Valgrind didn't exist. A certain species of programmer wouldn't fix memory stomping bugs in those days, nor leaks. They couldn't reproduce the problem on their PC, and they didn't have a tool that would just tell them where the problem was, so you'd just better accept that it's your lot in life to have crashes or leaks. They didn't want to buy the expensive (and frankly not very good) proprietary tools. So nothing to be done. Thankfully Julian Seward didn't agree. KMS needs a Julian Seward urgently.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 5:27 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]
That is overtly pessimistic. Especially the part about accusing developers of disinterest. As far as impressions go, I don't have to count on that or anecdotes. Reading bugzilla stats and the reports sent to test list show you how many bug reports have been filed and fixed via these test days and Xorg test days have been pretty successful. Anyway the point is about choosing the workflow that will be better than a general beta.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 10:20 UTC (Tue) by lmb (subscriber, #39048) [Link]
I don't think it is overly pessimistic. I'm afraid it is fairly accurate.
Trying to debug a graphics driver crash is non-trivial, and somewhat beyond the average Linux hacker. It certainly is beyond me; the hardware specs are closed, and the drivers and Xorg are not so good at diagnostic traces.
(Reporting the crash, and even getting one of the rare X backtraces just results in the very driver developers shrugging.)
And because of the way "direct" rendering works, a crashing application makes your entire hardware lock up. There's absolutely nil fault isolation.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 12:24 UTC (Tue) by nix (subscriber, #2304) [Link]
That tends to happen anyway. Graphics driver crashes have a nasty tendency to assert ownership of the PCI bus and then not let it go again. It doesn't matter what happened to the kernel when *that* happens: it's big-red-switch time.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 15:17 UTC (Tue) by lmb (subscriber, #39048) [Link]
So part of the problem seems to be literally hardware-related: software can't do fault isolation if the hardware doesn't. Wasn't this who (IO-)MMU thing supposed to help with that?
What prevents an OS from properly isolating a given driver, anyway? How does a gap here square with "virtualization", where we expect to be able to isolate guests from each other fully?
Maybe this is something to get right in the next version of hardware specs? The drivers will still suck, but at least they'll be easier to debug ...
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 15:58 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
> So part of the problem seems to be literally hardware-related: software can't do fault isolation if the hardware doesn't. Wasn't this who (IO-)MMU thing supposed to help with that?
Yes, but they are still not widely deployed on the desktop. I don't know to what extent Linux uses them if they are present.
> What prevents an OS from properly isolating a given driver, anyway? How does a gap here square with "virtualization", where we expect to be able to isolate guests from each other fully?
> Maybe this is something to get right in the next version of hardware specs? The drivers will still suck, but at least they'll be easier to debug ...
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 16:01 UTC (Tue) by mpr22 (subscriber, #60784) [Link] Parallel PCI has a rather nasty failure mode; I don't know if it's still there in PCIe. If a PCI device goes into a state where its interface is still capable of recognising transactions on the bus but its internal bus/switch/whatever is irretrievably wedged, it will sit there issuing Target Retry every time you try to access it. PCI bus master logic tends to be designed to wait basically forever for the Target Retry state to go away, so as soon as the processor issues a read to that device, or completely fills the host controller's posted-write buffer, the system will lock up. (And even if it doesn't lock up, it will crash.)
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 8:17 UTC (Wed) by cladisch (✭ supporter ✭, #50193) [Link]
I've seen something like this with a PCI device behind a PCIe/PCI bridge, where the north bridge issued a Machine Check Exception when the transaction timed out. (With a pure PCIe device, it would behave similar.)
The only difference between an immediate lockup and a MCE is that the kernel tries to output an error message before panicking.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 11:31 UTC (Tue) by interalia (subscriber, #26615) [Link]
<pedantry>
The word meaning "to lack interest" is "uninterested". To be "disinterested" means to lack bias. For example, you want a judge to be interested in what is said during the case but disinterested so that justice occurs.
It's a pretty common trip-up though.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 22:41 UTC (Tue) by mp (subscriber, #5615) [Link]
Apparently for every bit of such pedantry there exists a link to Language Log. This time it is http://languagelog.ldc.upenn.edu/nll/?p=511
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 12:29 UTC (Tue) by renox (subscriber, #23785) [Link]
> They didn't want to buy the expensive (and frankly not very good) proprietary tools.
Uh? I found purify to be a good tool.
> So nothing to be done. Thankfully Julian Seward didn't agree. KMS needs a Julian Seward urgently.
Well Valgrind's features were the same as those proprietary tools, is-there a proprietary tools which has the set of features you want for driver debugging?
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 13:36 UTC (Tue) by nix (subscriber, #2304) [Link]
valgrind has a substantially richer feature set than e.g. purify ever did.
a Valgrind for KMS Posted Sep 7, 2010 15:54 UTC (Tue) by tialaramex (subscriber, #21167) [Link]
AFAIU Julian saw a problem and approached it in a new way. Previously the accepted wisdom was that a decent memory debugger must be invasive, either instrumenting source or at least being linked in to mask out the system allocator. Valgrind's radical alternative is Virtualisation. The result is categorically more useful than tools like Purify in some circumstances, and the fact that it's GPL'd and thus can be included with the OS makes those circumstances all the more likely.
The accepted wisdom about KMS bugs, already repeated in this thread, is that if a device on a PCI bus (graphics chips have rarely been on a shared bus in the last decade but let's go along with the idea) goes haywire whether due to a hardware flaw or a driver bug, you're out of luck, it's unrecoverable. But it seems to me that most likely the CPU is still in charge here, so the situation is desperate, but not immediately fatal, just like a double fault. Very often it is technically possible to get things back into a working state, or at least to stash information about the circumstances of the fault and reboot - but the current drivers are oblivious so we hang.
Why mention Valgrind? Compare these alternatives:
1. "Sorry, I can't reproduce the bug you're describing. Maybe if you download the following sixteen projects as source, and rebuild them using [a memory debugging tool users can't reasonably be expected to own] you can collect enough information to fix it. Otherwise, sorry you're out of luck".
2. "Can't reproduce here. Do 'yum install valgrind'. Run the program with 'valgrind nameofprogram' and when it crashes paste the output into this ticket".
Guess which alternative leads to bugs getting fixed and which just teaches users not to even bother reporting them.
I'm not literally suggesting that there'd be a userspace binary that somehow debugs KMS drivers, but I am suggesting that the right smart person, possibly even the existing smart people who hack on these drivers given the inspiration, could make a dramatic difference to fix rate for KMS bugs by ensuring that they're debuggable. I am suggesting that this ought to be a very high priority, given the impact of unfixed KMS bugs (typically: the user has to run Windows or buy new hardware)
a Valgrind for KMS Posted Sep 7, 2010 17:51 UTC (Tue) by nix (subscriber, #2304) [Link] Valgrind's radical alternative is Virtualisation.Not unless you take severe liberties with the definition, it's not. Valgrind is a dynamic translator, a pluggable instrumentation engine, and a JITter. No virtualization as such involved (although of course the native CPU is not directly executing the valgrinded program anymore, so the CPU is virtualized in that sense, but nothing else is.)
if a device on a PCI bus (graphics chips have rarely been on a shared bus in the last decade but let's go along with the idea) goes haywire whether due to a hardware flaw or a driver bug, you're out of luck, it's unrecoverable. But it seems to me that most likely the CPU is still in charge hereEr, if the PCI bus has been grabbed by something else, I think you're doomed. This is the bus arbitration protocol that's gone wrong: there is no way to break a lock, no meta-arbitration protocol we can appeal to. (At least I can't recall one. If anyone knows differently, please speak up!) Equally, if the GPU has gone into an infinite loop and is not listening to the outside world anymore, even if the PCI bus is live you're not going to see anything on the screen. This is akin to an infinite loop in the kernel: all you can do is big-red-switch to get the system to listen to you again.
Does not work this way in Windows... Posted Sep 7, 2010 21:07 UTC (Tue) by khim (subscriber, #9252) [Link] The theory is sound, but the practice is different. Windows ATI driver can restart the card if it's stuck (usually from overclocking and/or bad cooling). I don't really know how it's done (perhaps there are some kind of hardware watchdog in GPU?) but it works. So even if theoretically it's possible to create hostile GPU which will hog the bus forever the real existing GPUs were not designed to work this way.
Does not work this way in Windows... Posted Sep 8, 2010 7:08 UTC (Wed) by mpr22 (subscriber, #60784) [Link] That parallel-PCI failure mode I described elsethread? I have encountered real existing hardware that could by incorrect operation be induced to cause that failure. Just because some hardware is sanely designed and well-implemented doesn't mean it all is.
Does not work this way in Windows... Posted Sep 8, 2010 10:04 UTC (Wed) by lmb (subscriber, #39048) [Link]
Conceivably, it would be an improvement if 80%+ of errors were easier to report. If there are still some that can't be - such as your PCI lockup - that is, of course, suckish, but getting the others fixed would help considerably.
Because it does make a difference if my system crashes every day, every week, or once per month.
a Valgrind for KMS Posted Sep 7, 2010 22:52 UTC (Tue) by tialaramex (subscriber, #21167) [Link]
"but nothing else is".
I hope I didn't mislead anyone, I didn't intend to. You are correct "only" the CPU is virtual. A valgrinded program writes to a real disk, connects over a real network, prints to a real printer. But its JMP instruction does not change the instruction pointer on a real CPU. That's where the magic happens.
a Valgrind for KMS Posted Sep 8, 2010 0:07 UTC (Wed) by nix (subscriber, #2304) [Link]
Well, not quite (though what you say isn't misleading, just simplified).
Because of the JITting, the JMP instruction *does* sometimes eventually end up as some sort of jump instruction in native assembler, and that *does* change the instruction pointer of the real CPU. (IIRC, sometimes valgrind may choose to run several JITted blocks together, so a JMP may be converted into nothing at all. But it's been a while since I looked at this.)
I'm not sure there's a term for the sort of on-the-fly there-and-back-again instrumented translation valgrind does. Paravirtualization, maybe, only that's already been used for something else. JIT is probably the best available term for it which isn't total compiler-hacker techspeak.
I wish there was a new valgrind paper that covered the fundamentals of the changes in it in the last five or so years. The earlier paper was one of the best things I read that year...
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 12:03 UTC (Tue) by alecs1 (guest, #46699) [Link]
More people would test if it was easy to test.
It is much much easier for me to run KDE 4.5 in Windows (I have it at work) than on my Debian box at home. Even more, I can install version 4.X for a user, and version 4.Y for another one. And all that with a few clicks, no special training.
The idea that software availability is less important than an "overblown" package system (with whatever benefits of security because libraries get upgraded and what not) is a loosing one on the desktop, particularly in this testing case.
Given the experience of wasted time, to get me testing you have to provide a program that does everything by itself (it should be an installer, rather than a script that compiles for 10 hours). I would have spent 3 hours testing 4.5 if it wasn't such a damn struggle to get 4.5.
Let's list the experiences here:
Oh, and I reported tens of bugs on KDE, and followed on each of them. I can't be accused I didn't try.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 12:13 UTC (Wed) by vonbrand (subscriber, #4458) [Link] Re 2: Rawhide packages are usually not signed (too much turnover), just do "yum update --nopgp".
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 12:21 UTC (Wed) by rahulsundaram (subscriber, #21946) [Link]
That would be
# yum update --nogpgcheck
kde-redhat.sf.net probably has the latest version available, backported for older releases. Safer than cherry picking from Rawhide.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 18, 2010 13:22 UTC (Sat) by jospoortvliet (subscriber, #33164) [Link]
Try using a liveCD?
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 21:55 UTC (Mon) by drag (subscriber, #31333) [Link]
> Even when taking a few steps back from the specific problems, it's noteworthy that KDE 4.x actually does not depend in OpenGL in any way - compositing itself is entirely optional to begin with, but the compositing manager also has a 2D-only XRender fallback that doesn't require OpenGL acceleration. This is different e.g. from Gnome 3's new Gnome Shell, which has a hard dependency on OpenGL (afaik, please correct me if I'm wrong here).
Metacity has optional Xrender-based composition in it. For the basic stuff Gnome-shell does I do not think that xrender would ever provide the functionality needed.
The solution here is not to force users to pick through features so that can work with their machines... The real solution is to make sure that the OS is able to provide the functionality that applications require. Of course application developers need to be reasonable... and I think that OpenGL 2.x support is reasonable.
Almost any desktop machine or laptop sold in the past few years has ample performance to be able to run a opengl-backed composited desktop. (those that do not, are not really suitable desktop machines and require special considerations) It's just a question of getting Linux up to snuff so that performance can be exposed to and applications and users.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 22:00 UTC (Mon) by elanthis (guest, #6227) [Link]
The important thing to note is that gnome-shell is planned to be optional, so gnome-panel, nautilus, and metacity can continue being used in desktops where the OpenGL stack is insufficient. Also note that gnome-shell (and kwin) may run decently with software OpenGL over the llvm/gallium stack.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 23:11 UTC (Mon) by renox (subscriber, #23785) [Link]
>> it's noteworthy that KDE 4.x actually does not depend in OpenGL in any way - compositing itself is entirely optional to begin with, but the compositing manager also has a 2D-only XRender fallback that doesn't require OpenGL acceleration. <<
And it's also noteworthy that apparently quite a few users don't agree with the setting they have (some find XRender too slow, some find OpenGL too buggy) and until Qt4.7 is deployed, you have to *recompile* to choose yourself the target used..
>> But then I also think that the desktops *should* push the envelope and try to exercise driver functionality that's in active development <<
That's funny: when Windows Vista was released, quite a few Linux users made fun about unneeded eye candy, now Linux desktops should exercise drivers????
IHMO desktops should be usable and stable by default for the biggest number of users, some additional "eye candy" effects would be provided (but turned off by default) and users feedback would be used to enable those feature by default if it works or to create bug reports toward drivers.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 23:25 UTC (Mon) by Sho (subscriber, #8956) [Link]
> And it's also noteworthy that apparently quite a few users don't agree with the setting they have (some find XRender too slow, some find OpenGL too buggy) and until Qt4.7 is deployed, you have to *recompile* to choose yourself the target used..
You're confused there. You're thinking of the painting backend used by the Qt toolkit (which can be set at run-time on a per-application basis, too, by the way, though changing the global default does indeed need a recompile), which is unrelated to the issue being discussed here, which is desktop compositing -- all of that code is in kwin, Qt is not a factor.
> That's funny: when Windows Vista was released, quite a few Linux users made fun about unneeded eye candy, now Linux desktops should exercise drivers????
OpenGL compositing is a feature high in demand by lots of users, so it gets worked on. Simple as that. Between KDE's OpenGL compositing support, Gnome-based distros choosing Compiz, and Gnome 3's OpenGL-based Gnome Shell, that seems to be a fairly universal conclusion among the mainstream desktop stuff.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 7:42 UTC (Tue) by renox (subscriber, #23785) [Link]
Thanks for correcting me for the Qt4.7 / kwin error.
>> That's funny: when Windows Vista was released, quite a few Linux users made fun about unneeded eye candy, now Linux desktops should exercise drivers????
I'd venture that having a desktop which 'just work' is also a feature which is in high demand by lots of users!
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 13:17 UTC (Tue) by tzafrir (subscriber, #11501) [Link]
KDE provides a desktop. For a system (including graphics drivers) that Just Works, ask distros.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 21:44 UTC (Tue) by renox (subscriber, #23785) [Link]
> KDE provides a desktop. For a system (including graphics drivers) that Just Works, ask distros.
The same distros that used KDE4.0 without warning users that it was a 'bleeding edge' version?
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 11:03 UTC (Wed) by mpr22 (subscriber, #60784) [Link] The fact that (some of) the distros screwed up over KDE 4.0 does not change the fact that putting together a set of software that will produce a system that Just Works is their responsibility. It raises questions over their competence to fulfill that responsibility, but it doesn't remove the responsibility.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 16:51 UTC (Tue) by sebas (subscriber, #51660) [Link]
> You're thinking of the painting backend used by the Qt toolkit (which can be set at run-time on a per-application basis, too, by the way, though changing the global default does indeed need a recompile)
Nope, that, as well can be set through an env var. You can recompile it to change whatever it chooses if the environment doesn't give any hint, but just changing a global environment var (QT_GRAPHICSSYSTEM), see http://doc.qt.nokia.com/4.7-snapshot/qapplication.html#se... (should be the same for 4.6).
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 17:39 UTC (Tue) by ABCD (subscriber, #53650) [Link]
I believe that the QT_GRAPHICSSYSTEM environment variable is new in Qt 4.7 -- previously, you had to either recompile Qt or run every Qt application with "-graphicssystem foo" on the command line.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 19:42 UTC (Tue) by aleXXX (subscriber, #2742) [Link]
No, it's not new, I remember I used it one or two years ago already.
Not quite sure which version of Qt it was, maybe 4.3 or something.
Alex
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 19:51 UTC (Tue) by Sho (subscriber, #8956) [Link]
Not quite. That env var has been in kde-qt/qt-copy for a while, but Nokia hasn't accepted that patch until Qt 4.7, so there's no stable release supporting it out there so far (though some distros like Fedora do carry the KDE patch).
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 9:45 UTC (Wed) by sebas (subscriber, #51660) [Link]
Ah, thanks for the correction. That's what I get for not running vanilla Qt ;-)
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 21:22 UTC (Mon) by ernstp (subscriber, #13694) [Link]
So create bugs for mesa, the kernel and the drivers in their respective bugtrackers and help the developers solve them!
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 21:25 UTC (Mon) by Sho (subscriber, #8956) [Link]
AIUI the blog post was aimed at KDE's users to explain what's going on right now, not an attempt to do bug-report-by-blog-post.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 22:51 UTC (Mon) by jrn (subscriber, #64214) [Link]
I believe that, but it is still helpful in such a post to link to bug reports. That would implicitly answer the question "well, what can I do about it?" that some readers may have.
Here are a possible few: https://bugs.freedesktop.org/show_bug.cgi?id=29148 https://bugs.freedesktop.org/show_bug.cgi?id=29223 https://bugs.freedesktop.org/show_bug.cgi?id=29711 https://bugs.freedesktop.org/show_bug.cgi?id=29873
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 9:28 UTC (Tue) by tcourbon (subscriber, #60669) [Link]
Hey look !
Their are one of my bug reports here !
*tickling his baby*
It's the first time I tried to report a bug to a such upstream project as xorg ddx. I double double checked the componenent I believed to be faulty. So I'm quite happy now as it look like I has spotted the culprit !
I know that the developers are by far too few to handle properly both the bug fxing and the claimed-since-so-long new feature implementation work. I want to help them and I would like to provide more information but I wasn't able to find directions for debugging the drivers (e.g : How could I trace the deficient function call, memory allocation or at least the faulty code path ?).
It's hard to find documentation generally speaking for those complicated piece of software. I dream of a graphicnewbies.org...
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 13:05 UTC (Tue) by cesarb (subscriber, #6266) [Link]
For Intel, the bug reporting guide is here: http://intellinuxgraphics.org/how_to_report_bug.html
I do not know if there is a similar guide for radeon or noveau.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 8, 2010 8:22 UTC (Wed) by tcourbon (subscriber, #60669) [Link]
Sadly I wasn't able to find something similar.
Dunno if it's just me being idiot or if it's that such guidelines didn't exist.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 11, 2010 6:18 UTC (Sat) by Thalience (subscriber, #4217) [Link]
If it is this one (guessing, based on your user name) https://bugs.freedesktop.org/show_bug.cgi?id=29873 then you did a pretty good job :) It would probably be helpful to add the version of Mesa that you are using, however.
Graphics corruption bugs are generally more difficult for end-users to generate good reports for than actual crashes, since there is no backtrace or anything. Perhaps KWin can be convinced to log the sequence of GL calls it is making, but I wouldn't know how...
Seeing as how you (or the reporter of that bug, if I guessed wrong) have an Evergreen (HD5770) card and public GL support for that card is only a few weeks old, the best way you can help is to keep testing the latest mesa git snapshots (running Arch should make this easy). Update (or close) the bug if the behavior changes for the better (or worse).
Developers always like to see that their work has fixed a bug, even if they weren't looking at your problem specifically. And they hate cleaning old bugs out of the bug tracker...
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 23:13 UTC (Mon) by renox (subscriber, #23785) [Link]
Application/desktop developer != driver developer.
And given the number of driver developers, those bug reports could take quite a few years to be solved, so the question is what should they do in the meantime?
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 6, 2010 23:53 UTC (Mon) by Thalience (subscriber, #4217) [Link] Bugs with good test cases are much less likely to sit in limbo than vague ones. For an application developer who doesn't want to dive into the GL stack internals, creating piglit test cases is probably the best way to help make progress on the bugs that matter to them.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 7:32 UTC (Tue) by drag (subscriber, #31333) [Link]
Your right. They should just continue to hack around crappy drivers from now to the end of eternity. Hack piled on hack making for a inconsistent, buggy nightmare for Linux users the world over.
I hope you noticed that despite the best efforts of the Kwin developer to put checks, hacks, and work-arounds in their window manager he still completely failed to write something that actually ended up working well.
And this is for a _window_manager_, one of the more boring and least interesting graphics applications for end users imaginable. Imagine what a game writer or 3D application developer has to deal with when targeting Linux.
Trying to hack around driver problems in applications is like trying to build a house on shifting sands: You are not going to succeed to do anything well and long-lasting no matter how much you try.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 12:17 UTC (Tue) by renox (subscriber, #23785) [Link]
> Your right. They should just continue to hack around crappy drivers from now to the end of eternity.
Eternity no, but given the pace of driver development, several years yes.
> Hack piled on hack making for a inconsistent, buggy nightmare for Linux users the world over.
Only if there isn't enough testing.
> I hope you noticed that despite the best efforts of the Kwin developer to put checks, hacks, and work-arounds in their window manager he still completely failed to write something that actually ended up working well.
I noticed, I also noticed that even if they wrote an infrastructure to work around driver issue, they failed to ensure that those driver would be tested correctly, as in:
Yes, it's a pain but notice that Linux kernel development team *disable* their new features by default, maybe they're wise?
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 7:09 UTC (Tue) by baburamjabi (guest, #69997) [Link]
KDE 5.0 Announced Today
Babu Kroger Ramjabi announced the next generation of KDE, forked with less than a month of coding by combining KDE 4.5 with Gnome 2.30.
How to do it? By forking and combining both desktops into one, with no testing required (like broken Gnome/Nautilus file manager).
Linux developers are well known for zero project testing, zero QA, and zero documentation. This is open source way, like the use of mailing-lists.
After a decade of development, KDE and Gnome have failed minimal 1% desktop market share. KDE 5.0 could break the 1% regression barrier.
As usual, KDE 5.0 includes Accessibility improvements, more Third World translations, and a new KSpeakEnglish website for the community.
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 9:32 UTC (Tue) by tcourbon (subscriber, #60669) [Link]
This is so not funny that it made me think you haven't get the point here. (Which by the way is pretty well commented in upper comments and in the article's blog post).
Graesslin: Driver dilemma in KDE workspaces 4.5 Posted Sep 7, 2010 9:35 UTC (Tue) by ovitters (subscriber, #27950) [Link]
Please take your trolls somewhere else. Thanks.
|
|||||||||||||