| |||||||||||||
LCA: The X-men speakLCA: The X-men speakPosted Feb 11, 2013 21:49 UTC (Mon) by marduk (subscriber, #3831)Parent article: LCA: The X-men speak
Reading this article made me think that maybe some of this work could be used to finally get decent "gnu screen"-like behavior in the X server, whereby if the X server can be disconnected from all GPUs then can keep running until one is re-attached/hot-plugged back in. That has always been my dream...
But when we finally have all that working we'll have to switch to Wayland :-|
(Log in to post comments)
LCA: The X-men speak Posted Feb 11, 2013 22:14 UTC (Mon) by marduk (subscriber, #3831) [Link]
I was aware of that project (well, the parent project) but my understanding is that it puts a virtual framebuffer between the X server and the clients and therefore does not take full advantage of the host GPU(s). But if the X server could "hot plug" GPUs then it could theoretically still keep the state of the X server running in the background, and then when a GPU is attached can talk directly to the GPU instead of a (slower) vfb layer.
LCA: The X-men speak Posted Feb 12, 2013 0:47 UTC (Tue) by samlh (subscriber, #56788) [Link]
The acceleration stuff is (for the most part) storing the window contents in textures on the GPU, and using the GPU to do the graphics operations. If you don't have a GPU, you cannot accelerate things. Theoretically one could move things on and off the GPU at hotplug time, but that is a recipe for worlds of pain - especially when you consider OpenGL and the like. I don't expect anything like this soon.
Practically, if you want to have screen-like behavior, you need a shadow framebuffer - or at the least a shadow 'windowbuffer'. This *will* have a performance impact, but it can be made reasonably fast. Indeed, the solutions mentioned in the other comments are quite usable.
LCA: The X-men speak Posted Feb 12, 2013 1:01 UTC (Tue) by samlh (subscriber, #56788) [Link]
Ok, I take at least part of that back - I now recall an GL extension (something to do with robustness) which allows applications to opt into being able to lose their state, and have them rebuild the gl context. With this, things move from impossible to merely difficult and vulnerable to app bugs.
Note, doing this in Wayland is likely to be no harder than it would be in X. Both would need to make the handles driver independent (possibly through a proxy driver), and both would need to track what is running on what GPU. The driver would need work as well, and they share the drivers.
LCA: The X-men speak Posted Feb 12, 2013 2:17 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
You can probably dump the current driver state and move everything to another GPU. That's something that COULD be done on the level of OpenGL implementation.
Though it's definitely not trivial.
LCA: The X-men speak Posted Feb 12, 2013 16:52 UTC (Tue) by daniels (subscriber, #16193) [Link]
What do you do when the two GPUs have different capabilities and extensions, as they almost always do?
LCA: The X-men speak Posted Feb 12, 2013 16:57 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
Restrict them to the common functionality. Which is not that bad in the Windows world, because Microsoft mandates the minimal functionality levels.
Also, the shim layer should always compile shaders with both GPU drivers at the same time so that you don't need to wait for too long during switches.
LCA: The X-men speak Posted Feb 12, 2013 16:58 UTC (Tue) by daniels (subscriber, #16193) [Link]
A large part of the reason for having two GPUs at all is that they'll expose different functionality, by virtue of one being more powerful (but more power-hungry), whilst the other is more efficient at doing less.
LCA: The X-men speak Posted Feb 12, 2013 17:02 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
Well, not really. GPUs mainly differ in raw speed - my integrated GPU has pretty much the same functionality as my discrete GPU.
Though there certainly should be a mechanism to get access to all GPU capabilities.
LCA: The X-men speak Posted Feb 13, 2013 9:48 UTC (Wed) by renox (subscriber, #23785) [Link]
>Well, not really. GPUs mainly differ in raw speed - my integrated GPU has pretty much the same functionality as my discrete GPU.
But not the other way round, correct me if I'm wrong but an integrated GPU can do 'page flipping'(and this is an important mechanism for performance) but a discrete GPU cannot do this because its memory is separated from the main memory.
LCA: The X-men speak Posted Feb 13, 2013 17:49 UTC (Wed) by daniels (subscriber, #16193) [Link]
Not entirely true: it depends how the buffers are allocated. Discrete GPUs are entirely capable of page flipping, as long as they can scan out of the memory.
The new mechanism Keith's talking about for partial page flipping (i.e. doing so on a per-page basis, whereas usually when we say 'page flipping' we mean doing it for the entire buffer - confusing I know) only really works at all on Intel hardware, and even then requires a great deal of co-operation from and hand-holding of all parts of the stack.
LCA: The X-men speak Posted Feb 21, 2013 10:27 UTC (Thu) by mmarq (guest, #2332) [Link]
"" only really works at all on Intel hardware ""
then is useless, for anything more than trivial.
LCA: The X-men speak Posted Feb 21, 2013 7:07 UTC (Thu) by elanthis (guest, #6227) [Link]
> GPUs mainly differ in raw speed - my integrated GPU has pretty much the same functionality as my discrete GPU.
Not true at all. It's very common for the discrete GPU to support a higher level of OpenGL and more extensions, which I really want to use. It's not at all uncommon on Windows to have self-written games and graphics apps crash and burn if you don't tell Optimus to use the discrete GPU for that process. If I know my game needs GL 3.3 and some extensions, and only one GPU can provide those, the stack should select that one. Likewise, the app probably knows if it only needs basic rendering or fast as possible rendering (the browser being a weird case, as different web apps might have different needs).
Part of the problem is that some heuristic and app list makes those decisions instead of the app saying during device handle creation that it would prefer the more capable GPU vs the more power efficient GPU. The app knows what it needs out of a GPU, and it should be able to at least hint to the stack about its preferences.
I know the DirectX/XDGI team has some plan here. No idea what Khronos plans to do with EGL, or if it's even thinking about the problem, but like most things Khronos does, it'll probably be some 1980's-style horrible error-prone API with zero debugging tools and be half a decade late to the party when it finally comes out.
LCA: The X-men speak Posted Feb 21, 2013 8:59 UTC (Thu) by khim (subscriber, #9252) [Link] It's not at all uncommon on Windows to have self-written games and graphics apps crash and burn if you don't tell Optimus to use the discrete GPU for that process. It used to be common problem, but is it really common here and now, though? Reviews of "third generation" HD Graphics (Ivy Bridge) usually highlight the fact that it's important milestone not because it's especially fast but because it finally bug-free enough to run most programs without crashing and artifacts. AMD's integrated graphics was always quite good in this regard. Part of the problem is that some heuristic and app list makes those decisions instead of the app saying during device handle creation that it would prefer the more capable GPU vs the more power efficient GPU. The app knows what it needs out of a GPU, and it should be able to at least hint to the stack about its preferences. Hint - yes, pick - no. Most GPU-using programs are proprietary ones which means that they are have the least amount of information in the whole system (user knows what s/he bought, OS can be upgraded but programs are written once then used for years).
LCA: The X-men speak Posted Feb 21, 2013 10:36 UTC (Thu) by dgm (subscriber, #49227) [Link]
> The app knows what it needs out of a GPU. and it should be able to at least hint to the stack about its preferences.
But it's the user who knows what he wants out of the app. It's the user's preferences for that app what matter.
LCA: The X-men speak Posted Feb 12, 2013 10:48 UTC (Tue) by lindi (subscriber, #53135) [Link]
On local machine xpra at least uses shared memory between client and server so performance is not bad at all.
LCA: The X-men speak Posted Feb 12, 2013 14:20 UTC (Tue) by marduk (subscriber, #3831) [Link]
> On local machine xpra at least uses shared memory between client and server so performance is not bad at all.
It does for specific use cases. If you are a developer coding on a 3D first-person shooter game in emacs then it's probably fine. If you then want to *play* that game, then I think shared memory is not enough. Moreover, what if you had one of those hybrid laptops and wanted to switch from the Intel GPU to the Nvidia one mid-play (although that may be possible now, but it sounds like maybe not). If/when the server can talk directly to the GPU *and* hot-plug them then that makes it easier do to things like this (easier albeit not easy) than if there is an intermediate server that can at best share memory with the server that owns the hardware.
The question though is what to do when there is no GPU attached and the clients are still running... perhaps you would need a virtual GPU layer or something like that.
Anyway that was my "dream". I realize that there are some solutions out there that kinda do what my dream is, but usually they do so inefficiently or require patches/workarounds for non-trivial use cases.
LCA: The X-men speak Posted Feb 12, 2013 16:20 UTC (Tue) by drag (subscriber, #31333) [Link]
The ultimate solution would be to not have the server do any of the rendering at all.
Instead of the server doing the rendering all the rendering is handled by the clients. The clients themselves talk to the GPU via whatever API they feel like using and then render their output to a off screen buffer.
Then all the 'display server' has to do at this point, besides handle the grunt work of doing things like handling inputs and modesetting, is flip the client buffer and composite the display.
With a little bit effort then it should be possible to kill the 'display server' and then have another 'display server' be able to grab the client application's output buffer.
Taking that and extending the idea further it should then be possible for you to take the client application's buffer it's rendering to, perform some sort of efficient streaming image compression, and then send that output over the network.
It may need slightly more bandwidth then traditional X, but it would still benefit from accelerated rendering (especially if your image compression can be accelerated) and would be virtually unaffected by latency... which is with X is it's Achilles's heal.
You could then have a single application on your desktop and then seamlessly transfer it to your phone or anything else running on any network. No special effort needed on the side of the application developer and you still get acceleration and all that happy stuff.
All theoretical at this point, of course.
/end wayland troll
LCA: The X-men speak Posted Feb 12, 2013 18:58 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
given that the clients run on the central box and the server is in front of the user, moving work from the server to the client concentrates the load, which is not a good thing (assuming that remote ability is maintained)
LCA: The X-men speak Posted Feb 12, 2013 20:42 UTC (Tue) by drag (subscriber, #31333) [Link]
yet oddly it's not much different then what X windows does when people try to use it to run 'thin clients'. The bulk of the application logic, memory usage, cpu usage, and the like always is going to be on the side of were the application is.
Is it going to be a big win for you to distribute the rendering of the applications?
It seems to me that the sort of applications were the bulk of the resources is put into the final rendering (video games, video playback, and the like) are exactly the sort of things that X11 is terrible at and is all done by direct rendering when possible.
LCA: The X-men speak Posted Feb 21, 2013 11:34 UTC (Thu) by mmarq (guest, #2332) [Link]
Sounds to me you have been reading about NX "drag"... (some ingenious stuff could apply not only remotely but also locally )
LCA: The X-men speak Posted Feb 11, 2013 23:35 UTC (Mon) by jonabbey (subscriber, #2736) [Link]
There's also FreeNX, OpenNX, and the commercial NoMachine NX.
LCA: The X-men speak Posted Feb 12, 2013 1:20 UTC (Tue) by smurf (subscriber, #17840) [Link]
Last time I looked, these depended on a severely hacked-up and out of date intermediate X server. Which is almost entirely unlike what you really want.
LCA: The X-men speak Posted Feb 12, 2013 11:39 UTC (Tue) by NAR (subscriber, #1313) [Link]
NX certainly provides screen-like access to remote servers (i.e. I turn off the local computer in the evening, turn in back on the next morning, reconnect and I get all my windows back in the same state) and in my experience it is surprisingly effective even on connections to servers thousand kilometres away. However, as far as I know, this has not much to do with the GPUs and I have no idea how much this depends on X servers.
LCA: The X-men speak Posted Feb 12, 2013 23:09 UTC (Tue) by jonabbey (subscriber, #2736) [Link]
NX does not depend on any sort of hacked X server, unless it is doing so under the covers, a la Xnest, which it may be doing.
I use OpenNX on Linux and Mac, and it has the best performance I have ever seen for remote application usage. What used to take 3 minutes over coffee shop wi-fi goes down to like 5 seconds with NX.
A great pity that it has gone proprietary for the latest version.
LCA: The X-men speak Posted Feb 13, 2013 1:26 UTC (Wed) by dtlin (✭ supporter ✭, #36537) [Link] NX does not depend on any sort of hacked X server, unless it is doing so under the covers, a la Xnest, which it may be doing. It absolutely is. nxagent is a fork of Xnest, although I'm not quite sure what version.
LCA: The X-men speak Posted Feb 21, 2013 11:06 UTC (Thu) by mmarq (guest, #2332) [Link]
" A great pity that it has gone proprietary for the latest version. "
NX started as proprietary... yet
So the mechanism (protocol addition) is free... still is... but blindness has a price... why maintain a free version when there is no "support"... i would do the same, improvements only on the proprietary "version"...
And the funny thing is that NX seems to address many of the issues that were discuss about wayland... most could be used not only on the context of remote display, but local also...
http://www.linuxjournal.com/article/8477?page=0,2
* an efficient compression of what normally would be X traffic
* an intelligent mechanism to store and re-use (cache) data that is transfered between server and client
* a drastic reduction in multiple, time-consuming X roundtrips, bringing their total number *close to zero*.
So it seems clear to me its not a copy or derivative of Xnest.. at all..
And i couldn't think of a better addition to the all X protocol than this... but sometimes wonder if Xorg has any say, or only does what its told to do, and that is why it has been "striped" and now positioned to be "finished"... hope the distros could take a serious look at this situation and take over, hire a "Ninja" or two, fresh ideas, specially the desktop oriented ones... their future is in jeopardy.
|
|||||||||||||