Weekly Edition
Return to the Development page
| |||||||||||||
St. Pierre: The Linux Graphics Stack
Jasper St. Pierre has posted a lengthy
overview of the Linux graphics stack. It's a good starting point for
anybody who is not clear on what all those acronyms mean. "The X
server needs to know what’s happening here, though, so it can do things
like synchronization. This synchronization between your glxgears, the
kernel, and the X server is called DRI, or more accurately, DRI2. 'DRI'
stands for 'Direct Rendering Infrastructure', but it’s sort of a strange
acronym. 'DRI' refers to both the project that glued mesa and Xorg together
(introducing DRM and a bunch of the things I talk about in this article),
as well as the DRI protocol and library. DRI 1 wasn’t really that good, so
we threw it out and replaced it with DRI 2."
(Log in to post comments)
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 15:39 UTC (Fri) by dps (subscriber, #5725) [Link]
I fairly regularly run the odd client remotely, most frequently xload. In a former life I was almost always running xpbsmon remotely and I suspect many users of a cluster still do so.
In a similar vein the Mac OS X version of M$ office would be much more useful if I could run them remotely on my main screen, which is much larger than my Macbook Pro's display. I usually use {Open,Libre}Office and fall back on M$ office when these don't work.
Clouds, in which an application will often run remotely, have just made a network transparent GUI even more vital. If wayland loses that then it is a major step backwards. OTOH if wayland makes remote applications very fast, like NoMachine's NX can, then it would be a major win.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 16:30 UTC (Fri) by renox (subscriber, #23785) [Link]
> if wayland makes remote applications very fast,
It's not the case AFAIK, Wayland is for the local case and can be extended for the LAN but I really doubt that it will be good on a WAN (we'll see)..
> like NoMachine's NX can, then it would be a major win.
The major win would be to have NX integrated in X (as an extension) so that it would be available everywhere, but this didn't happen, and I doubt that it will in the future: very few people work on X.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 18:28 UTC (Fri) by gebi (subscriber, #59940) [Link]
Network transparency in Wayland would on the same level as rdesktop under Windows (afaik).
So it should finally provide a usable remote desktop experience under Linux. And not some slow as hell blob... (try starting eclipse, chrome or Matlab with X-forwarding).
I really hope wayland will just implement rdesktop, so we finally have a unified remote desktop program, and not the current hacked together *VNC solutions, all incompatible without real encryption/auth/... (with non opensource realvnc being the only usable option, or rdesktop).
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 2:08 UTC (Mon) by butlerm (subscriber, #13312) [Link]
The RDP protocol with the full suite of Microsoft extensions is far more extensive than anyone apparently intends to implement with Wayland. In particular it remotes a full alpha composited rendering API similar to XRender, a 3D protocol comparable to GLX, and object caching with differential update comparable to NX.
Even with those extensions turned off, the abstractions that it supports are comparable to X itself, except (apparently) without the design mistakes that make X / Xlib essentially unusable on a high latency network. Of course it is a considerably more recent design with protocol updates on a regular basis.
With Wayland as it stands, I don't see how anyone can implement remote access that is much more sophisticated than VNC. At a minimum, performing substantially better than VNC would require that windows be treated separately, and be composited and managed on the client (display server) - rather than with a bitmap level export of the entire desktop.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 2:20 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
>With Wayland as it stands, I don't see how anyone can implement remote access that is much more sophisticated than VNC. At a minimum, performing substantially better than VNC would require that windows be treated separately, and be composited and managed on the client (display server) - rather than with a bitmap level export of the entire desktop.
That's exactly how Wayland remoting is (probably) going to work.
Now, accelerated composition _within_ windows is another question.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 20:47 UTC (Fri) by raven667 (subscriber, #5198) [Link]
I should point out that what NX actually does is similar to how a Wayland remote app would need to work so Wayland should actually be faster because there is less hoop-jumpery in the process.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 1:59 UTC (Sun) by jonabbey (subscriber, #2736) [Link]
NX does a lot more than rdesktop can with its client side caching of x resources. NX performs better for us at low bitrates than any other solutions we've tried.
NX is like rsync for remote display tasks.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:08 UTC (Sun) by raven667 (subscriber, #5198) [Link]
Yep the point I was making is that NX is a full X11 proxy, it renders everything on the app side then ships rendered data across the network with caching and whatnot on the display side. It is a different implementation of the same concepts that underlie most other remote display systems like RDP, ICA, PCoIP or SPICE. The same concepts, with probably a different implementation will certainly come to Wayland because there is such a demand for it.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:18 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
that differs from my understanding of NX. I understand that NX is a X11 proxy, but that it does a lot to eliminate roundtrips, not that it does the rendering and then ships bitmaps.
As I understand it, NX is very similar to WAN accelerators that companies buy to make using CIFS to remote sites better, they fully understand the protocol and use their knowledge of what the remote side would send in reply to generate that reply without having to send the request.
both X11 and CIFS are excessively chatty, requiring a lot of round-trips. When you add latency to these round trips, performance drops significantly, even if you have good bandwidth, by short-circuiting many of the round trips, performance drastically improves.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 6:38 UTC (Mon) by butlerm (subscriber, #13312) [Link]
>NX does a lot more than rdesktop can with its client side caching of x resources
NX doesn't really cache client side resources so much as differentially encode new messages in terms of old ones, which has much the same effect.
Recent versions of RDP, on the other hand, don't appear to do much in the way of inter-message differential compression, but provide the option to cache and re-use a bewildering number of resource types on the client (display) side. That seems rather inflexible (i.e. applications practically have to be programmed that way) compared to what NX does, however. See here:
Remote Desktop Protocol: Composited Remoting V2 Specification
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 16:34 UTC (Fri) by gmaxwell (subscriber, #30048) [Link]
I'm pretty sure there isn't an "if" on wayland not having network transparency. There appears to be complete consensus on that point from everyone working on it.
This is also coming from/will be adopted by the same development culture (though not the same people) who think it's okay to introduce reboot-for-upgrades. It's a focus on the narrow desktop(/tablet) most tightly own the marketplace. I think its sad because competing with their companies on their own terms is a losing proposition. But ::shrugs:: How other people spend their time isn't my call.
I wonder how much of the disinterest in network transparency and the belief that frame grabbing (for this use) remote video solutions like VNC and spice can replace it is created because a lot of the newer toolkits are really inefficient in their server communications.
I frequently run older X11 apps remotely over wireless by launching something in the wrong shell and not realize that I'm on a remote system. It's a lot harder to make that mistake with heavy cairo using applications (for example, I don't mean to suggest that its alone)... It's still much faster to use remote X over low latency networks than things like VNC, but there is less of a difference.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 22:02 UTC (Fri) by daniels (subscriber, #16193) [Link]
> I'm pretty sure there isn't an "if" on wayland not having network transparency. There appears to be complete consensus on that point from everyone working on it.
You're completely correct. The consensus is that we will have support for remote clients.
And, speaking as someone who's currently working on Wayland and has been working on X for the last ten years, I can't see how it couldn't be better. X over the network is disgracefully poor.
But hey, fact seems to be totally orthogonal to any discussion about X and Wayland on LWN.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 22:27 UTC (Fri) by gmaxwell (subscriber, #30048) [Link]
> You're completely correct. The consensus is that we will have support for remote clients.
Then educate us please, because all I can recall seeing is the position that Wayland is that remote would only be supported through "pixel-scraping" "like VNC"— which is exactly how systems that have no concept of remote displays work, and which— as done so far— has pretty poor performance on low latency networks.
I'd certainly love to hear otherwise!
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 22:36 UTC (Fri) by daniels (subscriber, #16193) [Link]
> Then educate us please, because all I can recall seeing is the position that Wayland is that remote would only be supported through "pixel-scraping" "like VNC"— which is exactly how systems that have no concept of remote displays work, and which— as done so far— has pretty poor performance on low latency networks.
That's what X does too. Look at any app using GTK+ or Qt, and you'll notice that they render absolutely everything on the client side and just send huge precomposed images the size of the window up to the server. This has been true for years now. So, the fundamental model does not change one bit from X11.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 23:28 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
as long as the only apps that you ever use are GTK+/QT, and those toolkits never get fixed, this would be true.
but is it really the right thing to do to make those two assumptions?
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 4:50 UTC (Sat) by smoogen (subscriber, #97) [Link]
Hasn't every toolkit done this though? [Having watched networks go down in the day from running various browsers on X terminals.. I know both Motif and the basic Xtoolkit widgets suffered from it when you threw anything like a large gif across the network.]
My understanding since the early 1990's was that the large data blobs going over the network can't be fixed without fundamental rewrites of the X stack itself.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 12:22 UTC (Sat) by daniels (subscriber, #16193) [Link]
'fixed' is an odd choice of word.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 13:56 UTC (Sun) by ebassi (subscriber, #54855) [Link]
and what, pray tell, would need to be "fixed" in GTK+ and Qt?
replacing high quality rendering with, what? X drawing primitives? the 1987 called, and it wants its graphics system back.
also, for added fun, try to convince people writing web rendering engines to ditch client-side rendering and compositing to implement the W3C spec, and just use X drawing primitives instead.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 21:43 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
the equivalent here would be to convince all the web developers to ditch CSS and all client-side javascript in favor of having the server render all images and just sending the bitmaps to the browsers.
everywhere else the trend is to push more of the display workload towards the edge where there are lots of relatively powerful devices, but for some reason when you start talking about X/Wayland people talk about making the layer closest to the display dumber and doing more of the work on the other end.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 21:47 UTC (Sun) by daniels (subscriber, #16193) [Link]
I agree, it's ridiculous and indefensible that the graphics stack developers haven't pushed the entire GL stack into the kernel.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 21:58 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
this has nothing to do with in-kernel or not.
in games, in websites, even in local displays, the trend is to have the application do less of the display rendering and push more of the rendering closer to the display (in the game client, in the javascript/CSS, and in the video card respectivly)
In games, this is why cheats make it possible to let you see through walls, the game has pushed the rendering to the client, so if the client is modified the player can get information that they could not get if the server was only telling the client what to display
but when we start talking about remote desktops, suddenly we are told that it's obvious that the only thing to do is to ship bitmaps around, anything else has been proven to be too horrible to tolerate?
If this is true, why is it that the 'modern desktop' somehow has higher requirements than any games? because the games all want to have the display (or system the display runs on) do more of the work
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:14 UTC (Sun) by daniels (subscriber, #16193) [Link]
Maybe it's a sign that doing remoting at the buffer management level, rather than the toolkit level, is some kind of poor idea?
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:22 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
> Maybe it's a sign that doing remoting at the buffer management level, rather than the toolkit level, is some kind of poor idea?
exactly, which is why so many of us are saying that the Wayland approach of doing remoting at the buffer level is such a bad idea.
I don't think anyone is claiming that X11 is the perfect toolkit for doing remoting, but whose of us who do use remoting routinely are concerned that the X replacement doesn't include any concept of remoting at the toolkit level.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 23:20 UTC (Sun) by daniels (subscriber, #16193) [Link]
By your same logic, that Wayland doesn't subsume Apache and Chromium is also some kind of fatal design flaw.
It sounds like we agree that remoting should be done at the toolkit level. Your argument seems to be that Wayland should also be a toolkit, which I think is a terrible, terrible, terrible idea.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 23:55 UTC (Sun) by neilbrown (subscriber, #359) [Link]
The picture I'm getting is that Wayland isn't trying to replace X11. It is trying to replace just part of X11 - but that part is currently the most widely used and so the best understood. It is being designed so that those other bits of X11 can be added via other systems which use the facilities provided by Wayland without having to fight with wayland.
There are lots of ways to do remoting, so Wayland doesn't provide any. However any system that tries to add remoting can focus on just moving the images/events across the network without worrying too much about how to put them on the display - Wayland does that
Remoting can be done by sending images over the network using various compression schemes, or by sending javascript to update a canvas, or by sending X11 protocol requests which update windows in the same way they always have. Or maybe even using display-postscript.
With the Wayland approach, each of these can co-exist as equals, can be independently developed and can compete with each other for different market segments.
Wayland does seem to exclude the possibility of running the window manager over the network, but I cannot imagine anyone really wanting to do that. Even the thinnest of thin clients from 20+ years ago had the option of a local window manager
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 16:47 UTC (Mon) by nix (subscriber, #2304) [Link]
I'll be happy with remoting in the toolkits -- as long as they all do it via some shared library or something like that, so that at the very least something like $DISPLAY is consistently respected. It would be appalling to have to tell each toolkit separately where its display was, any authentication and so on (though it is probably best to let ssh handle the authentication and encryption side of things these days).
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 0:32 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
Currently X11 is the remoting toolkit.
Wayland is claiming that it's the replacement for X11.
If this is true, then Wayland needs to be, or provide the remoting toolkit. Or the Wayland developers need to be designing wayland to interact with the separate remoting toolkit.
Instead Wayland advocates are saying that remoting will be done in wayland by shipping bitmaps around similar to how VNC does it, they then go further and say that is the only sane way to do remoting, and by the way, remoting is obsolete and not a desirable feature anyway.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 2:02 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
>Currently X11 is the remoting toolkit.
Currently X11 is a *failed* remoting toolkit. FTFY.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 8:57 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
for something that's such an epic fail, a lot of people are getting good use out of it.
nobody is saying that it's perfect, not by any means, it's just better than anything else currently available (in part due to it's large installed base and good backwards compatibility)
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 14:25 UTC (Mon) by HelloWorld (guest, #56129) [Link]
> for something that's such an epic fail, a lot of people are getting good use out of it.
The sole reason people use X.org is that it contains the drivers needed to make their hardware do anything. When this started to change (i. e. introduction of KMS, evdev, EGL and all that), people immediately started working to finally get rid of it.
> nobody is saying that it's perfect, not by any means, it's just better than anything else currently available
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 2:35 UTC (Mon) by alankila (subscriber, #47141) [Link]
Remoting the X way is a feature that is sacrificed to get other goods, such as the ability to not define rendering in Wayland's display protocol. As explained elsewhere, this will allow rapid evolution of clients, and a degree of future-proofing as nonexistent rendering protocol will not become obsolete either.
Technically the compositor gets notification from the client about new pixels, so it is in position to do something like break the updated region in 8x8 tiles and check out if they match previously remoted tiles, and thus only send cache key instead of the data to allow cheap form of bandwidth reduction which should fit well for most of our GUIs. The downside is that it does force the compositor to do pixel scraping to discover what is different since last time, and that part imho may need help to reduce CPU usage of the Weston server and to improve efficiency of tile caching (if this is the way it will be done).
Maybe some hints should be provided, like "this region is video and will change every frame, just compress and send it" and "this area was scrolled by (x, y) pixels before updating, so you probably want to apply an offset in your tile-matcher." etc. These should be viewed as purely performance optimization to improve remoting for some specific workloads such as scrolling a viewport or watching video, both which are liable to generate tremendous amount of network traffic almost no matter what.
St. Pierre: The Linux Graphics Stack Posted Jun 26, 2012 4:32 UTC (Tue) by drag (subscriber, #31333) [Link]
There are lots of ways to deal with it.
For example Spice is able to heuristically determine if a portion of the screen is displaying video and then will use mjpeg to transmit that to the client.
I would expect that at a minimum if you are expect wayland itself to be network transparent then you would just transmit portions of the visual buffer that changes. If a window contents have not updated then there is no reason to send a update. If just some text or some portion of the window has updated then there is no reason to send the entire buffer again.
If the window portion is not visible then no reason to send a update. If you are moving a window around then there is no reason to send a update.
All sorts of stuff like that.
A lot of this stuff is not theoretical. All you have to look at is something like ICA or RDP to find something that is much more widely used and performs much better then X ever did.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 16:12 UTC (Mon) by nix (subscriber, #2304) [Link]
I just dug back through all your comments on this point that I could find on LWN. Every single one of them was nothing but mockery. I'd have expected a Wayland and core X developer to have more substantive points than the equivalent of 'nyaah nyaah', really...
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 21:56 UTC (Sun) by gioele (subscriber, #61675) [Link]
> the equivalent here would be to convince all the web developers to ditch CSS and all client-side javascript in favor of having the server render all images and just sending the bitmaps to the browsers.
There is actually a disturbing trend towards getting rid of CSS and declarative HTML and replacing those with canvas + JavaScript. WebGL is another example of that trend. I'd say that all these trends parallels that from X11 to Wayland.
Anyway, I do not dislike the idea of having a fancy framebuffer API (Wayland) on top of which place a more updated declarative graphical API (X12 let's say). The problem with that does not lay in Wayland, but in the fact that we are losing a declarative API to draw things on the screen (we actually lost it years ago when we started using client-side fonts) in favour of toolkits like Qt and GTK+. Maybe an extended OpenVG will fill this void, maybe HTML will.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:02 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
> There is actually a disturbing trend towards getting rid of CSS and declarative HTML and replacing those with canvas + JavaScript. WebGL is another example of that trend. I'd say that all these trends parallels that from X11 to Wayland.
I see them as being completely the opposite, the equivalent of wayland would be to stop having the browser doing the rendering and having the web server create the bitmaps and send them to the browser.
moving to more Javascript + canfas, or WebGL is pushing more of the rendering out to the edge, closest to the display
For Wayland, people are saying that you don't want the smarts at the display, you want to ship bitmaps around to the display, having the other end (server in web terms, client in X terms) doing all the rendering.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:21 UTC (Sun) by raven667 (subscriber, #5198) [Link]
This is a rediculous comparison, the web is a client/server architecture with the client apps automatically downloaded and cached on the client and written in JavaScript. It's not like remote displayed apps at all. You are so clearly comparing apples and oranges here that your statement doesn't add anything to the conversation but is merely a distraction from it.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:45 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
a point I am trying to make is that everything is moving to a client-server architecture.
with local displays, the apps are doing less of the rendering themselves, and offloading more of the work to the video card.
with websites, the apps are doing less of the rendering on the server side and pushing more to the browsers.
with games, the central servers are doing less of the rendering, pushing more of the work to the clients.
with displayport, even the link from the video card to the display is becoming more client-server like, pushing more smarts to the display rather than just providing a stream of pixels like everything before it has done.
everywhere you look the trend is to segment out the work, doing less of it in what has been the core application, and hand as much of the work as possible off to other components, Those components may be separate software processes, or they may be hardware components, but more of the work is being pushed out towards the user.
so I don't believe that remoting a desktop should be an exception to this trend. and I especially don't believe the statements that there is something inherently wrong with a smart protocol when compared to shipping bitmaps around.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 8:27 UTC (Mon) by renox (subscriber, #23785) [Link]
> with games, the central servers are doing less of the rendering, pushing more of the work to the clients.
That's not strictly true, there has been some talks of 'gaming in the cloud' where the servers do all the computations and send bitmaps to the display (like Wayland will do), of course it doesn't exist currently because first one must solve bufferbloat and even then it's not sure that the result will be good enough.
That said, I agree with you, shipping bitmap around is OK for a LAN but will probably suck on a WAN.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 9:52 UTC (Mon) by Jonno (subscriber, #49613) [Link] > That said, I agree with you, shipping bitmap around is OK for a LAN but will probably suck on a WAN. Well, it depends on the WAN in question, X11 (and even NX) are more latency-sensitive while Xpra and Wayland are (will be) more bandwidth-sensitive. For example, over an 8 Mbps ADSL line (moderate latency) NX performs better than Xpra, but over an 6 Mbps HDSPA connection (high latency) Xpra performs better. Essentially, as long as the bandwidth isn't saturated, Xpra (and in the future Wayland) will perform better (e.g. be more responsive), but once the bandwidth is saturated, it depends on the complexity of the application. In most cases NX will perform better, but in some cases (i.e. when total texture size >= final bitmap size) Xpra/Wayland will perform better).
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 11:44 UTC (Mon) by cortana (subscriber, #24596) [Link] Do you mean services such as OnLive? They are already in production, and work quite well. Sufficiently well enough that even someone as cynical as me was impressed.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 16:31 UTC (Mon) by Cato (subscriber, #7643) [Link]
Gaming in the cloud (game streaming) does indeed exist, with OnLive and Gaikai being two leading players - Gaikai has a Java client so it runs fine on Linux clients and is worth a try compared to the hassles of getting only a limited set of PC/Windows games to run on Linux via WINE (been there, done that).
I used OnLive for a while and it's quite usable, although the graphical quality was really fairly poor on a 3-4 Mbps connection. Being able to just play a game for 30 minutes before deciding to buy it is really handy, and generally it's nice to be able to play a game on any (Windows) PC.
Game streaming is just a special case of remote desktop access of course - OnLive also provide access to Windows desktops in the cloud, which would be useful for Windows to Linux migrations if they had a Linux client.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 17:35 UTC (Mon) by butlerm (subscriber, #13312) [Link]
> and what, pray tell, would need to be "fixed" in GTK+ and Qt?
Presumably, GTK and Qt could be changed to rely on XRender and any other necessary extensions when they are available, instead of pre-compositing everything itself.
XRender seems like an adequate, modern alpha composited rendering API. If it isn't it should be fixed. That plus frame barriers to avoid tearing should be enough to do it right.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 18:18 UTC (Mon) by ebassi (subscriber, #54855) [Link] Presumably, GTK and Qt could be changed to rely on XRender and any other necessary extensions when they are available, instead of pre-compositing everything itself. this is what happens already. it turns out, though, that none (except older intel ones) of the modern GPUs are optimized for XRender's usage of trapezoids; as every GPU is, essentially, a programmable 3D pipeline, triangles are actually more suited for them. in point of fact, XRender is not at all a good drawing and compositing API. the proper drawing and compositing API is, actually, OpenGL. that is also why Cairo, which was born as the client-side XRender API, is now moving to GL internally, and its internals are in the process of being overhauled for that.
St. Pierre: The Linux Graphics Stack Posted Jun 26, 2012 17:09 UTC (Tue) by butlerm (subscriber, #13312) [Link]
>none of the modern GPUs are optimized for XRender's usage of trapezoids
Is it really a serious problem to convert trapezoids into a pair of triangles, or is the problem simply that trapezoid based rendering requires too many trapezoids period?
St. Pierre: The Linux Graphics Stack Posted Jun 26, 2012 21:56 UTC (Tue) by ebassi (subscriber, #54855) [Link]
the tessellation algorithm is completely different in case of triangles; trapezoids provide a different set of benefits and complexities (basing XRender on trapezoids was not done on a whim). and if you remove trapezoids tessellation then you basically chucked away the majority of XRender - so you might as well go with another API entirely.
plus, XRender has nothing to describe vertex shaders, or geometry shaders, and even the convolution filters are a poor version of the GL fragment shaders.
in practice, OpenGL provides a better API to talk to modern GPUs than XRender.
St. Pierre: The Linux Graphics Stack Posted Jun 27, 2012 21:15 UTC (Wed) by butlerm (subscriber, #13312) [Link]
>OpenGL provides a better API to talk to modern GPUs than XRender
Using OpenGL to do 2D graphics strikes me as first class overkill. But if it is the only thing that a GPU can actually do well, then fine. I don't know why anyone at the application or toolkit levels would want to write OpenGL code to do 2D graphics rendering though, if there were any possible way they could avoid it.
The issue with remote access is worse - relying on a serialization of OpenGL on the wire to handle 2D over a low bandwidth channel would be insane, as in not fit for the purpose at all. As an actual wire protocol, XRender ought to be much better, simply because it is much simpler, much more stable, and much less likely to break.
However, if there is some sort of gratuitous impedance mismatch between XRender and the way GPUs like to see things, perhaps the designers of the next generation remote graphics rendering protocol could take that into account. I am curious what could be done in any case - it isn't at all obvious why using triangles instead of trapezoids for 2D graphics is going to make anything better. The source material is Bezier curves after all, not some sort of polygonal mesh.
St. Pierre: The Linux Graphics Stack Posted Jun 29, 2012 12:33 UTC (Fri) by ebassi (subscriber, #54855) [Link] Using OpenGL to do 2D graphics strikes me as first class overkill. that's because you still assume that: a) there is such a thing as a separate 2D and 3D and b) that this fictitious separation applies to the hardware. both assumptions are entirely fallacious. 2D is just a special case of 3D; modern GPUs are just programmable 3D rendering pipelines, and OpenGL is just a vendor-neutral API to program them without using driver-specific API: it provides you with the API to compile and upload programs to the GPU, as well as uploading geometry and texture data to the GPU memory - that's it. not only the separation of 2D and 3D does not exist anywhere near the software layer, it's also not in the hardware: there are no modern GPUs with separate 2D pipelines like they existed in the past - except some Intel stuff. I don't know why anyone at the application or toolkit levels would want to write OpenGL code to do 2D graphics rendering though I maintain a toolkit that uses OpenGL. Cairo has a GL rendering surface that allows you to use the device-independent API to draw high quality 2D content using GL. Qt does the same. ideally, you don't want app developers to fudge around with GL - it's an awful API, with lots design-by-committee-of-CAD-developers crap - unless you're a game developer. that's why people write toolkits on top of it, exactly like people wrote toolkits on top of Xlib. this has happened on other platforms as well: CoreAnimation and CoreGraphics on MacOS and iOS are two APIs used to do 2D (and 2D-layers-in-3D-space) UIs based on OpenGL and OpenGLES; CoreAnimation is the base toolkit used to write the whole user interface on iOS. Windows has its own toolkit based on DirectX. Android moved from software rendering to hardware acceleration through GLES. tl;dr: the world has changed in the past 10 years - both at a hardware and a software level. GPU manufacturers standardised on GL and DirectX as the API exposed to control their cards, and platform and UI design moved to those two as the preferred drawing API for implementing their user interfaces.
St. Pierre: The Linux Graphics Stack Posted Jun 29, 2012 17:27 UTC (Fri) by daenzer (✭ supporter ✭, #7050) [Link]
The problem isn't trapezoids vs. triangles per se but that the XRender rasterization rules (and some other aspects of its semantics) don't match what GPUs can do natively (or ever could, even when XRender was invented). On modern GPUs, the XRender rasterization can be emulated in pixel shaders, but older GPUs require at least the rasterization to be done by the CPU.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 23:46 UTC (Fri) by gmaxwell (subscriber, #30048) [Link]
I did mention the newer stuff being much worse and I suggested this as a possible reason that there was a reduction in interest in supporting doing it well anymore.
I'm now confused as to why you'd said the comments on wayland were disconnected with the facts, since it sounds like we agree on the facts.
Shipping around images of the windows/screen is what you do when you have no concept of network transparency and it has generally bad performance (after all, even 24fps 1080p RGB is over a gigabit/s of raw data). I think its perfectly fair to say that a system that depends on shipping images of the is not one with integrated network transparency. You can define things otherwise, but if you do then end up with a definition of network transparency that doesn't permit the existence of a non-network transparent system. Is this really just some argument over terminology?
Fine. I don't know what network transparency is then. Instead, one thing I have historically liked about X is Weeblix: The ability to remotely run an application across a lan with performance which is close or indistinguishable from local. VNC, Rdesktop, and Spice do not provide Weeblix, because they're all obviously slow even over a gigabit network. Plain X11 stuff does even when tunneled through ssh at least over low latency networks (e.g. spin up gnuplot's x11 terminal remotely— perhaps do a 3d splot and spin things with the mouse). I understand that wayland's design is not very weeblix friendly and that wayland will not be making an effort to provide weeblix.
Maybe systems which are fluidly and painlessly usable across a lan are dead, but I'll still morn their loss... Or really, they're actually the norm— but we now call them webapps and basically none of them are free software and they usually stink in entirely different ways (slow, vanish when their creators lose interest, etc). Oh well.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 12:24 UTC (Sat) by daniels (subscriber, #16193) [Link]
If you want to continue running X11, then I recommend you keep doing so. X is 27 or so years old by now, and the appearance of Wayland isn't going to make it disappear overnight.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 14:49 UTC (Sat) by drag (subscriber, #31333) [Link]
> Maybe systems which are fluidly and painlessly usable across a lan are dead,
I don't know why you say that.
The use of remote desktop applications and thin clients by consumers and businesses have exploded in the past few years.
It's just none of them are using X to do it.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 23:46 UTC (Fri) by nix (subscriber, #2304) [Link]
Seems to me that the server needs *more* intelligence, not less. Sending giant precomposed bitmaps is clearly insane: the only reasons Gtk and Qt do it are that it reduces tearing (because there is no way to tell the server that particular things must be rendered simultaneously or not at all: no 'render barriers' if you will) and that it allows forms of higher-order operations that X cannot implement because its protocol cannot be extended by code uploaded to the server by the clients. The general rule should always be that computations that must affect images with low latency should be done on the same machine as the user if at all possible. (Obviously this is not possible in all cases, but, again, we're trying to speed up the common case, not someone applying an expensive GIMP effect. The user doesn't expect *that* to be fast in any case.)
Honestly, DPS got half this right twenty years ago. It just chose a horrible server-side rendering language, and existed almost purely on a dead platform, so nobody used it.
I don't understand why everyone ignores the possibility now. The server-side code need have no more privileges than the client: you could even run it in a distinct process or set of processes to enforce that.
But, no, giant bitmaps it is. Sigh.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 3:46 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]
We have much better server-side display language. It's called 'HTML'.
It even has integrated Turing-complete language (something called 'JavaScript') for small server-side applications to reduce number of round-trips!
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 12:28 UTC (Sat) by daniels (subscriber, #16193) [Link]
> Seems to me that the server needs *more* intelligence, not less.
No way, no how.
So let's assume you've managed to code the entirety of GTK+ within the Wayland protocol, and the themes and everything. You've got your nice rounded corners, your shiny reflective bits, etc. Even once you've managed to do that, all that means is that your compositor is now trying to perform all the rendering for 27 clients.
Suddenly your entire system is hopelessly bottlenecked on one process which is trying to do all the rendering for every clients, and is absolutely enormous. The latency for your _entire system_ has just gone totally out the window because your compositor's going to be blocked on applying CSS to complex layouts. Cool.
Now imagine doing the same but with two toolkits. Or maybe three. Because hey, X11 is about choice, right?
Then what happens when you want to change your toolkit? You're pretty much screwed, because it's encoded in the compositor. Either that, or you just end up doing what GTK+ and Qt have been doing for the last n years, and shipping precomposed images.
So yeah, one thing that X11 taught us is never to do that in the server. If you want great GTK+ remoting, then you might do well to write a GTK+ proxy or something. But Wayland isn't a toolkit, and never will be. It takes images that clients have given it and puts them on the screen and nothing else.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 15:10 UTC (Sat) by paulj (subscriber, #341) [Link]
If you want great remoting though, my experience is that the old, X-drawing-primitive using lib/applications had noticeably better (i.e. remained useable) performance over network links than the modern "send pixmaps" using lib/apps have. Now, sure, the chattiness of X does require a fairly low-latency link. However, that affected both types of rendering. Indeed, as bandwidth commonly decreases as latency increases, for many kinds of real-world links the high-level-primitive apps tended to remain significantly far more performant than the ship-a-pixmap ones did, even on higher latency links.
Also, it's not clear to me that the chattiness problem of X is intrinsically related to the high-level-drawing-primitive model. I.e. where VNC manages to cope better with latency, is that inherently because it uses a ship-a-pixmap model, or is simply because it has a different, more efficient event reporting/interaction model to X?
It'd be nice to be able to read a thorough exposition on the performance implications of high-level-drawing primitive versus ship-a-pixmap, with respect to bandwidth and latency, from those involved in Wayland and/or any others who've been involved in remote GUIs.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 18:24 UTC (Sat) by drag (subscriber, #31333) [Link]
> If you want great remoting though, my experience is that the old, X-drawing-primitive using lib/applications had noticeably better (i.e. remained useable) performance over network links than the modern "send pixmaps" using lib/apps have.
I suspect you are viewing your past experiences through rose colored glasses.
The modern versions of GTK, for example, do quite a bit of work to hide the bad aspects of X11 when doing remote applications.
Some simple benchmarks of Xterm versus Gnome-terminal.
$ xterm -v
Xterm is running the x11 fonts. Gnome-terminal is running Dejavu Sans Mono with default settings (I think anti-aliased and all that junk).
F17 to CentOS:
Debian to CentOS:
And on top of that the gnome-terminal does a much better job being responsive. When I run the find and hit 'ctrl-c' the find on gnome-terminal exits immediately while on xterm it will continue to run for a few seconds before stopping.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 20:12 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]
heh, I've been finding myself moving to older terminal programs (xterm, rxvt, etc) instead of the gnome or kde terminals because even on local boxes, the gnome and kde terminal programs are noticeably more sluggish
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 22:39 UTC (Sat) by paulj (subscriber, #341) [Link]
Yeah, initial startup always took longer for the high-level primitive based apps. Takes many round-trips to paint window on top of window. The "send rendered pixmap" apps win on that. It was after that that the HLP apps would win on responsiveness, istr.
The find/ctrl+c test you're doing is, I suspect, not demonstrating anything about high-level primitives v send-a-pixmap. Rather, I think, that's somehow down to different buffering & display of terminal output (namely, gnome-terminal is just way more optimised now - there was a time when it was atrociously bad in this situation compared to xterm I think).
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 22:55 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]
Oh, it was not only initial startup. Simple window resizing takes _ages_ on old X-apps.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 13:44 UTC (Sun) by nix (subscriber, #2304) [Link] I.e. where VNC manages to cope better with latency, is that inherently because it uses a ship-a-pixmap model, or is simply because it has a different, more efficient event reporting/interaction model to X?The latter. One thing that is indubitably wrong with X is the number of roundtrips the protocol requires, despite all the in-Xlib caching to keep that down. It may be the case that it is impossible to keep roundtrips down while keeping intelligence up... but I suspect this is not true, that you only end up with a roundtrippy protocol if you try to have a protocol that describes low-level operations and also provide no way to encode higher-level operations on a client-by-client basis on the server side. But, hey, one nice thing about Wayland is that it has so little intelligence that if I ever have the time to experiment with this I can do it on top of Wayland!
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 21:47 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
we know that the major problems with X have to do with the number of round trips.
There are even extensions that improve this drastically (including NX), the problem is that the basic X11 setup isn't able to assume that these extensions are available. If we were to simply define X12 as being X11 with these extensions being mandatory instead of optional there would be a significant improvement (and if we could get the NX folks to release their improvements so that they could be made part of the mandatory stack it would be even better)
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 21:48 UTC (Sun) by daniels (subscriber, #16193) [Link]
And yet it'd still be massively, massively flawed.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:23 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]
You're talking about NX as if it was some magic pixie dust.
NX works by pre-rendering as much as possible on the client side and shipping pre-rendered instructions to the display server. It can do efficient compression since it knows how images are composed (i.e. it can cache individual anti-aliased glyphs, rounded borders, etc.), but other than that it's not that much different from VNC.
xpra works similarly, btw.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 3:13 UTC (Mon) by butlerm (subscriber, #13312) [Link]
> NX works by pre-rendering as much as possible on the client side and shipping pre-rendered instructions to the display server
May I ask where you obtained this information? Everything I have read about NX is that it is a highly optimized differential X protocol compressor, which of course wouldn't normally be expected to do anything of the kind.
See here, for example:
xpra, on the other hand, is completely different, and it certainly sounds reasonable to implement something similar to xpra within Wayland itself, because the xpra protocol and the Wayland application interface are based on essentially the same abstraction - transfer and compositing of window framebuffers. An xpra like protocol is exactly what I expect someone to implement for Wayland first - it is the simplest, easiest and most obvious thing to do.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 3:35 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]
>May I ask where you obtained this information? Everything I have read about NX is that it is a highly optimized differential X protocol compressor, which of course wouldn't normally be expected to do anything of the kind.
That's a description of how it effectively works with modern GTK/Qt applications. NX works fast because it can cache most of image drawing operations used to create graphical QT/GTK primitives. And it's deteriorating fast, QT5 apps are very slow over NX links because they do quite a lot of drawing on the client and simply upload bitmaps.
Compression of the X protocol command stream itself is not that significant.
There's an overview here: http://www.nomachine.com/documents/NX-XProtocolCompressio...
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 6:13 UTC (Mon) by butlerm (subscriber, #13312) [Link]
> NX works fast because it can cache most of image drawing operations used to create graphical QT/GTK primitives
It certainly caches messages containing such operations, and differentially encodes new messages based on messages previously sent, but that isn't the same thing as doing any kind of pre-rendering.
St. Pierre: The Linux Graphics Stack Posted Jun 26, 2012 18:51 UTC (Tue) by hp (subscriber, #5220) [Link]
The protocol doesn't require a lot of round trips, it's Xlib that does.
With xcb (or hacky xlib hacks for example http://git.gnome.org/browse/metacity/tree/src/core/async-...) you can pretty much write code with zero round trips. At litl we wrote a pretty complex compositing OpenGL WM with apps integrated into it, and squished all the roundtrips out of it.
Maybe it's more precise to say that the protocol doesn't require _blocking_ round trips. There are 'getters' with a reply, but you can batch up 100 requests and then harvest 100 replies, all nonblocking. Also most 'getters' have to do with properties (and protocols like clipboard) rather than with rendering.
With indirect GLX, the problem becomes bandwidth, rather than roundtrips, even on a local socket, because the way indirect rendering works is that every GL API call is a remote message. It doesn't block but it just adds up to a bunch of stuff to push on every frame if you're trying to have a decent framerate. (We were trying to make this work on an Atom.)
On modern desktops you can get away with lots of inefficiency locally, but I don't think indirect GLX could ever be very fast over a network.
St. Pierre: The Linux Graphics Stack Posted Jun 26, 2012 21:35 UTC (Tue) by nix (subscriber, #2304) [Link] The protocol doesn't require a lot of round trips, it's Xlib that does.I was alluding to the fact that a major raison d'etre of Xlib was to keep roundtrips down (by doing exactly the sort of batching that you describe). However, it is perhaps too opaque, making it hard for users to detect that they're about to require a roundtrip... and, of course, it is less efficient than more recent attempts (such as yours), and the code is nasty enough that fixing it is more trouble than it is worth. (Fixing it without breaking its contract with callers is probably impossible anyway.) Indirect GLX, well, I wish it still worked, but it's stuck at OpenGL 1.4, and last time I tried it on a DRI2 system it failed to function entirely, even for something trivially simple like glxgears. Nobody expects it to be as fast as local rendering (at least I hope nobody does), but if you require insanely fast local rendering for an OpenGL rendering of a 2D desktop something is wrong, I'd say. I don't really expect remoting to work for full-motion video, or full-screen 3D games, or something like that.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 13:41 UTC (Sun) by nix (subscriber, #2304) [Link] Suddenly your entire system is hopelessly bottlenecked on one process which is trying to do all the rendering for every clients, and is absolutely enormous.I clearly said that you could run the server-side programmable rendering in subprocesses (or threads, or whatever, though subprocesses seem smarter to keep locking hell down). I also mentioned that this would only be practical if the code were somehow uploaded from the clients: it obviously won't work if you have to hardwire it all into the server. HTML and JS are doing this already, as others have pointed out, so it is neither rocket science nor impossible. But you were too busy pooh-poohing the idea based on a quick skim to actually read what I said. Fine.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 2:40 UTC (Mon) by butlerm (subscriber, #13312) [Link]
> let's assume you've managed to code the entirety of GTK+ within the Wayland protocol, and the themes and everything.
That is a bit of a red herring - no one here is suggesting remoting at the toolkit level. As you say, that would likely have all the problems of HTML + CSS, except worse.
The question of the day is whether there really should be a remotable rendering layer that lies in between the toolkits and the windowing system. The only reasonable answer I have heard for why Wayland isn't planning on one is that it would be a big project comparable to re-designing X from scratch, which is correct.
Toolkits themselves have internal APIs like this however that they use to implement multiple backends. So while it would be a big project, it would seem to me that it would basically amount to developing a common denominator between the GDK backend API and the new Qt (Lighthouse) backend API. Then GDK / Qt could support this new shared API as separate backends, the new API could be implemented in-process for fast local performance (on top of Wayland in particular), but allow remoting protocols to plug in at the drawing API level rather than bitmap level.
No one is going to design a GDK specific or a Qt specific remoting protocol. Whatever is done, the remoting standard must be able to support arbitrary toolkits, and the only general purpose way to do that is with a common remotable drawing API that multiple toolkits can support. Not instead of Wayland, but on top of Wayland, between Wayland and the application toolkits.
Client vs Server Intelligence Posted Jun 24, 2012 1:29 UTC (Sun) by ldo (subscriber, #40946) [Link] nix: Seems to me that the server needs *more* intelligence, not less. Sending giant precomposed bitmaps is clearly insane... Nope. The accumulated empirical evidence over about the last 30 years has shown conclusively by now that having an intelligent rendering server is a bad idea. Much better to have your rendering/display/print server be completely dumb and deal with nothing but bitmaps rendered on the client side. That simplifies the protocol, leaves far fewer opportunities for things to go wrong, and just as importantly, means that nearly all feature upgrades only need to happen at the client end, they don’t have to wait for the developers of the server end to drag their feet because few clients support the new features, so the client developers in turn drag their feet because few servers exist to show off the new features.
Client vs Server Intelligence Posted Jun 24, 2012 9:06 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
and this is exactly what all those so called 'smart' video cards that take rendering instructions have died out and been replaced by simple framebuffers that just display the bitmaps that the applications send to the video card, just like the experts at Sun predicted so many years ago.
oh wait, it's the dumb framebuffers that are dieing off.
why is it that local framebuffers are so hard to find, but you say that it doesn't make sense for the user's display to be anything more than a dumb framebuffer?
Client vs Server Intelligence Posted Jun 24, 2012 9:32 UTC (Sun) by HelloWorld (guest, #56129) [Link]
What does this have to do with video cards? Just because the rendering is done on the client side doesn't mean it doesn't happen on the video card.
Client vs Server Intelligence Posted Jun 24, 2012 10:08 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]
why should a rendering interface to a video card on the system generating the image be a good thing, but a rendering interface to a remote machine that's displaying the image be a bad thing?
the idea that you would use a rendering interface to a local video card, then extract the resulting bitmap across the bottleneck of the local system bus to then send the bitmap across the network
vs
sending rendering commands to a remote system to then pass to it's video card
seems rather inefficent.
I'm not saying that the selection of drawing primitives offered by X are the right thing, but it seems from everything I'm reading that X with available extensions (especially with NX) works very well, the main problem being that these extensions are all considered 'optional'.
It would seem to me that especially with the x.org monoculture, it should be possible to define these extensions as being standard, or at least define that some new software won't work without the extensions rather than throwing out the established protocol entirely and moving to something completely new like wayland.
Client vs Server Intelligence Posted Jun 24, 2012 13:03 UTC (Sun) by alankila (subscriber, #47141) [Link]
I personally would give up on GPU-driven rendering in a heartbeat if we could do the same amount of rendering on CPU. Unfortunately, our CPUs appear to be significantly too slow for the task at hand.
Client vs Server Intelligence Posted Jun 25, 2012 9:27 UTC (Mon) by etienne (subscriber, #25256) [Link]
> Unfortunately, our CPUs appear to be significantly too slow for the task at hand.
Sometimes I wonder if the CPU is fast enough but the cache subsystem is really not made for efficient screen display: usually you enable the memory cache on video memory, so to display a pixel the CPU has to *read* the rest of the cache line - and even on PCIe that takes time (i.e. not optimised for reads). The video memory read is even needed if you will overwrite next pixel very soon, the "write and invalidate" PCIe transaction seems not often used. Unfortunately I do not have real numbers, I just note that the GPU do not have data caches.
Client vs Server Intelligence Posted Jun 25, 2012 18:12 UTC (Mon) by alankila (subscriber, #47141) [Link]
While that is an interesting statement in itself, in my head I was mostly referencing the times it takes to do operations like memory copies, alpha blending, bilinear texture filtering and (maybe) running user-defined programs per pixel of output. A high-end GPU has around 1000 cores today, and I imagine many of them have performance that is some relatively large fraction of a typical desktop CPU single core's performance.
That people *can* nevertheless turn off all GPU acceleration and get usable performance for some simple tasks is nice, but it also seems to be a fact that it is not possible to watch a fullscreen video or animate windows on screen or do transparency with fluid frame rates without GPU taking that work off the CPU's hands.
Client vs Server Intelligence Posted Jun 25, 2012 19:03 UTC (Mon) by raven667 (subscriber, #5198) [Link]
I think this whole conversation thread is off in the weeds trying to compare general purpose CPU cores to the types of cores that GPUs contain, they are so different as to not be directly comparable. They are designed to run different kinds of software entirely, a GPU can't run general purpose software, a CPU can run GPU software but doesn't have the highly parallel resources to do so quickly.
Client vs Server Intelligence Posted Jun 27, 2012 2:04 UTC (Wed) by alankila (subscriber, #47141) [Link]
Yes, but that is exactly the core reason why we have GPU doing the rendering, so maybe it still relates to why we use them. I am basically saying they are a necessary evil, not necessarily a great design from an abstract point of view.
Client vs Server Intelligence Posted Jun 28, 2012 9:31 UTC (Thu) by etienne (subscriber, #25256) [Link]
> a CPU can run GPU software but doesn't have the highly parallel resources to do so quickly.
I fully agree, but I do not see that many highly parallel resources needs.
Client vs Server Intelligence Posted Jun 24, 2012 13:47 UTC (Sun) by nix (subscriber, #2304) [Link]
This is clearly true if you have to hack the server to support every new feature. But you don't, as long as the server supports running sandboxed uploaded code. (I thought I said as much but nobody seems to have read it. Perhaps I am hallucinating.)
Client vs Server Intelligence Posted Jun 25, 2012 2:58 UTC (Mon) by butlerm (subscriber, #13312) [Link]
> The accumulated empirical evidence over about the last 30 years has shown conclusively by now that having an intelligent rendering server is a bad idea.
That is not correct. RDP as actually implemented by Microsoft is a major counterexample, as is ICA, and of course NX. In fact virtually every graphical application remoting protocol that has reasonable performance over a high latency network appears to be a counterexample.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 1:43 UTC (Sat) by magcius (guest, #85280) [Link]
No. They use XRender, and send trapezoids over to the server. I mentioned this in the article.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 23:32 UTC (Fri) by obi (guest, #5784) [Link]
While I do realize that indirect 3D rendering has lost most of its usefulness these days, but it'd be nice if there was at least the possibility for using remote 3D with wayland in the future.
Or is there really no point in that any more?
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 16:40 UTC (Fri) by drag (subscriber, #31333) [Link]
> In a similar vein the Mac OS X version of M$ office would be much more useful if I could run them remotely on my main screen,
On my Windows workstation at work I run the majority of my applications remotely and seamlessly. The application launchers are integrated into the start menu and access to applications are regulated by Active Directory group policy.
About the only applications that actually run locally are my web browser and putty.
These applications are ran out of a separate data center then the one I administrate and they run much faster and much better over that connection then X11 applications running on Linux workstation local over the lan. It's bad enough that I don't even bother with X11 networking unless it's behind a Nomachine NX server.
Really doubt that 90% of the people in the organization realize that their windows applications run remotely, except the call center folks which rely entirely on thin clients (Wyse terminals running Suse to display Citrix Windows systems, I believe)
Right now Microsoft Windows has proven that it's quite possible to start off with a Windowing system that was not originally designed specifically for network apps and yet is able to coaxed into performing massively superior to X.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 17:49 UTC (Fri) by Darxus (guest, #85281) [Link]
Network transparency is inevitable, which is obvious to everyone involved. It's expected to work better than X, because of X's pathological round trips (which wayland handles *much* better), and the fact that most images are basically only outputting an image buffer anyway - nothing is using stuff like server side line drawing or or font rendering of X anymore.
The bug to implement it is here: https://bugs.freedesktop.org/show_bug.cgi?id=48981
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 19:46 UTC (Fri) by krakensden (subscriber, #72039) [Link]
> nothing is using stuff like server side line drawing or or font rendering of X anymore.
I thought everyone *was* still using X's font rendering? Or at least the glyph caching?
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 19:54 UTC (Fri) by magcius (guest, #85280) [Link]
There's a few "font rendering protocols". XRender has a way to cache different kinds of glyphs for an easy copy. See the render protocol, section 12:
http://cgit.freedesktop.org/xorg/proto/renderproto/plain/...
Darxus is talking about things like XFLDs and XmbDrawString. Nobody uses those.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 20:24 UTC (Fri) by apoelstra (subscriber, #75205) [Link]
See this screenshot segment from my display (I am using wmii, which is essentially the same as dwm, but with easier configuration options):
http://download.wpsoftware.net/images/22jun2012.png
You can see that the Firefox window has antialiased text, using some Gtk+ magic, while the window decorations do not. I believe that is what magicus was referring to.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 20:26 UTC (Fri) by apoelstra (subscriber, #75205) [Link]
Sorry, I thought you were replying to a different part of the thread. Please ignore my message.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 20:59 UTC (Fri) by renox (subscriber, #23785) [Link]
Wayland has also its own round trips induced by the client-side decoration design(*), for example when moving a window..
*:but at least KDE devs plans to use Wayland and server side decoration.
St. Pierre: The Linux Graphics Stack Posted Jun 24, 2012 22:37 UTC (Sun) by raven667 (subscriber, #5198) [Link]
I think this might be backwards but I could be wrong. I thought the client side decorations _reduced_ the need for round trips because there is no need to coordinate the drawing of the border with the drawing of the application, it can all be done in one shot and the compositor just sees a rectangle to place. In the current X11 scheme performance of window moving and resizing isn't as good as it can be because of all the IPC and context switches needed to coordinate drawing the border with drawing the app contents with the compositor and window manager processes. The borders can be drawn offset from the contents which should never happen.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 7:36 UTC (Mon) by renox (subscriber, #23785) [Link]
1) It's very likely that X is suboptimal for remote rendering otherwise NX wouldn't exist.
2) Wayland's client's side decoration management *do create* some additional RTT compared to an ideal server side decoration&window management: think about moving windows:
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 8:37 UTC (Mon) by dgm (subscriber, #49227) [Link]
> -with a server side decoration & window management, moving window can be done only in the server.
This is a common misunderstanding. There are no server-side decorations in X11, because the Window Manager is NOT part of the server, it's just an special client.
> -with Wayland's current design, Weston has to send the mouse clicks to the client as it is the client which will interpret those mouse clicks as a move request and then send to Weston a 'move my window' request.
That's correct, but compare it with equivalent X11:
The user clicks some point of the decorations, the X11 server(1) handles the event to the window manager(2) that then sends some movement command back to the server, that re-routes it to the compositor(3), and finally comes back to the server. That's 3 processes (and a lot of switches) instead of just 2. Even if you don't use a compositor, it's still 2 processes, the same as Wayland.
And the resize case is still worse, because the application, that was conspicuously absent in the previous case, has to do some drawing itself, adding still more round trips. That adds visual artifacts unless the compositor keeps the image off-screen, in which case it adds latency to the feedback the user gets, making window movement more jerky.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 8:54 UTC (Mon) by renox (subscriber, #23785) [Link]
>> -with a server side decoration & window management, moving window can be done only in the server.
> This is a common misunderstanding. There are no server-side decorations in X11, because the Window Manager is NOT part of the server, it's just an special client.
'server side' means for me: on the same machine as the display.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 14:01 UTC (Mon) by dgm (subscriber, #49227) [Link]
Wonderful, then everything is server side in Wayland. Latency heaven! :-P
St. Pierre: The Linux Graphics Stack Posted Jun 29, 2012 17:33 UTC (Fri) by daenzer (✭ supporter ✭, #7050) [Link]
For anyone who believes Wayland won't help for window resizing or even make it worse, I recommend actually trying it out with Weston on KMS. It's rather eye-opening! Mac OS X grade snappiness and smoothness. 8)
As someone who's been working on X related stuff for more than a decade, I'm looking forward to the day I can run my desktop session on Xwayland, if not directly on a Wayland compositor.
St. Pierre: The Linux Graphics Stack Posted Jun 29, 2012 18:05 UTC (Fri) by hummassa (subscriber, #307) [Link]
I don't get it. Or I didn't parse something here.
I just opened one non-maximized Chrome window in one Kubuntu Precise desktop, and another one in an OSX desktop. Moved them. Resized them. Repeated with Firefox. Repeated with Amarok and iTunes. Except for the fact that in KDE, my moving and resizing have the transparency effect, there is absolutely no tearing nor anything that bothers me. It's exactly the same responsiveness that I have on OSX. Care to explain?
St. Pierre: The Linux Graphics Stack Posted Jul 3, 2012 16:24 UTC (Tue) by daenzer (✭ supporter ✭, #7050) [Link]
Did the window contents always match the window size as well without lag? I've never seen that nearly as good as Mac OS on any X setup, certainly not on the machine where it was the case with Weston. But if it's already the case with X on your setup, good for you. :)
St. Pierre: The Linux Graphics Stack Posted Jul 3, 2012 20:00 UTC (Tue) by hummassa (subscriber, #307) [Link]
I am trying to come up with a better example, but I did just open systemsettings (KDE's control panel) and resized it wildly and the icons jumped from place to place (there is no animation for that... I will test it out in OSX when I get home b/c my laptop is not here) but all was perfectly smooth. No lag, no tearing, no flickering.
(KDE 4.8.2 on Precise on an onboard Intel graphics chip for Dell Optiplex 790 machine, but it seems to be this way since... forever, i don't know)
(actually, there is a little bit of tearing if you turn the more "eye candy" effects like the "deform windows while moving")
URI for video (trying to upload to G+ b/c seems faster):
https://plus.google.com/u/0/100671974217756568906/posts/Y...
St. Pierre: The Linux Graphics Stack Posted Jul 4, 2012 12:41 UTC (Wed) by hummassa (subscriber, #307) [Link]
Update: undistinguishable (modulo the translucency) from what happens in OSX; opposite to what I recalled, the icon movements are similar... tested in Finder/OSX and in Dolphin/KDE because the OSX control panel isn't resizeable!
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 16:50 UTC (Fri) by apoelstra (subscriber, #75205) [Link]
A very well-written, clear article. But the author makes comments like
>Even basic features and innovations like “Click to Focus” that were in Windows 3.1 for Workgroups hadn’t caught on yet. The popular environment at the time, CDE, had a system called “focus follows mouse”, which focused windows when the user moved their mouse over the window itself.
which seem very condescending and out of place. Many people use focus-follows-mouse (including myself -- I also use a tiling WM) and find it to be much superior to click-to-focus. With the same attitude, he describes some crufty aspects of X and introduces Wayland. He explains the job of the compositor as
>These window managers, more accurately called “compositors” in Wayland terms, are actually in charge of pulling events from the kernel with a system like evdev, setting up a frame buffer using KMS and DRM, and displaying windows on the screen with whatever drawing stack they want, including OpenGL. While this may sound like a lot of code, since these subsystems have moved elsewhere, code to do all of these things would probably be on the order of 2000-3000 SLOC. Consider that the portion of mutter just to implement a sane window focus and stacking policy and synchronize it with the X server is around 4000-5000 SLOC, and maybe you’ll understand my fascination a bit more.
I can't quite grok this, but I'm pretty sure he's saying that the entire Weston compositor is around 2-3KLOC, compared to the tens of KLOC used to implement the corresponding parts of the X subsystems. But his comment about mutter is rather disingenuous -- dwm, for example, is a complete X window manager in ~2200 lines.
To run such a beast under Wayland, the dwm folks would need to write an entire compositor to talk to the kernel and manage the framebuffer, in direct conflict with the project's goal of having an easy-to-grok, easy-to-hack WM.
Perhaps somebody more knowledgeable than me could answer: Would it be possible to write a Wayland compositor that will talk to X window managers, so that xmonad/wmii/dwm/whatever will work without serious modification?
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 17:39 UTC (Fri) by magcius (guest, #85280) [Link]
> which seem very condescending and out of place. Many people use focus-follows-mouse (including myself -- I also use a tiling WM) and find it to be much superior to click-to-focus.
Sorry, it wasn't meant to be condescending, only show that X11 itself cannot itself enforce a basic policy for focus management. I've updated the tone of the article to respect that a bit better.
> I can't quite grok this, but I'm pretty sure he's saying that the entire Weston compositor is around 2-3KLOC, compared to the tens of KLOC used to implement the corresponding parts of the X subsystems. But his comment about mutter is rather disingenuous -- dwm, for example, is a complete X window manager in ~2200 lines.
That's what I'm saying. Was the language unclear?
dwm is an extremely simple window manager. It uses core X drawing protocols to draw its UI, even for drawing text, which don't have antialiasing. It doesn't do compositing.
Again, to set up the appropriate features for a modern desktop environment is as much code as it would be to write dwm, which is an extremely simple window manager.
> To run such a beast under Wayland, the dwm folks would need to write an entire compositor to talk to the kernel and manage the framebuffer, in direct conflict with the project's goal of having an easy-to-grok, easy-to-hack WM.
You don't have to use libdrm. You could use OpenWF or EGL, which manage that part for you.
> Perhaps somebody more knowledgeable than me could answer: Would it be possible to write a Wayland compositor that will talk to X window managers, so that xmonad/wmii/dwm/whatever will work without serious modification?
It's possible, but it would be difficult. At the same time, you'd have to implement a lot of features of the X, so you're better off running Xorg or Kdrive, and putting it under Wayland, like Xwayland.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 18:08 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
>> which seem very condescending and out of place. Many people use focus-follows-mouse (including myself -- I also use a tiling WM) and find it to be much superior to click-to-focus.
>Sorry, it wasn't meant to be condescending, only show that X11 itself cannot itself enforce a basic policy for focus management. I've updated the tone of the article to respect that a bit better.
What you see as a failure (X cannot enforce a policy) is seen by many others as a feature (you don't have to try and work around whatever X is enforcing and can instead implement whatever you want)
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 18:31 UTC (Fri) by magcius (guest, #85280) [Link]
I really don't see it as a failure. I see it as a compromise in history, which complicates behaviour a lot.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 3:36 UTC (Sat) by drag (subscriber, #31333) [Link]
I don't see why wayland compositor enforcing policy is mutually exclusive with having the ability to change the policy if you don't like it.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 19:01 UTC (Fri) by apoelstra (subscriber, #75205) [Link]
>Sorry, it wasn't meant to be condescending, only show that X11 itself cannot itself enforce a basic policy for focus management. I've updated the tone of the article to respect that a bit better.
It looks much better now, thanks.
>> I can't quite grok this, but I'm pretty sure he's saying that the entire Weston compositor is around 2-3KLOC, compared to the tens of KLOC used to implement the corresponding parts of the X subsystems. But his comment about mutter is rather disingenuous -- dwm, for example, is a complete X window manager in ~2200 lines.
>That's what I'm saying. Was the language unclear?
If you explicitly gave an estimate for how many lines of code X requires for these things, it'd be clear, I think. As it is, it takes a bit of thinking to understand what you're comparing to what.
Otherwise, this was a very clear and informative article, and cleared up a lot for me!
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 20:04 UTC (Fri) by Zizzle (guest, #67739) [Link]
> Sorry, it wasn't meant to be condescending, only show that X11 itself cannot itself enforce a basic policy for focus management. I've updated the tone of the article to respect that a bit better.
Does that imply that Wayland will enforce this policy and the policy will be click-to-focus?
If so, Wayland will be a non-starter for me.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 21:17 UTC (Fri) by magcius (guest, #85280) [Link]
No. It's up to the compositor to enforce a policy now.
St. Pierre: The Linux Graphics Stack Posted Jun 22, 2012 17:43 UTC (Fri) by Darxus (guest, #85281) [Link]
> To run such a beast under Wayland, the dwm folks would need to write an entire compositor to talk to the kernel and manage the framebuffer, in direct conflict with the project's goal of having an easy-to-grok, easy-to-hack WM.
No, it can be written as a plugin to weston: http://www.chaosreigns.com/wayland/plugins
There was an early attempt at a tiling wayland compositor called ADWC: http://www.phoronix.com/scan.php?page=news_item&px=MT...
OpenGL over the network? Posted Jun 22, 2012 23:51 UTC (Fri) by pflugstad (subscriber, #224) [Link]
If I understand correctly, a Wayland client would connect to the Wayland server and get a buffer "handle" that it would share with the Wayland server and the kernel itself. All OpenGL commands will be done by the Mesa/Gallium3d (possibly hardware accelerated) directly on this buffer handle. Then when the OpenGL glFinish/glFlush gets hit, the Wayland client notifies the Wayland Server which composites the clients buffer to the the "real" screen buffer. Am I tracking what's going on?
For a remote client, this buffer would be local to it, and the glFinish/glFlush would trigger the resulting buffer to be sent over the network to the Wayland server which would then composite it to the screen. So you could have different OpenGL hardware acceleration capabilities on the client vs the server?
To me, this seems like it must very inefficient - no matter how smart you are, copying a big buffer of data can't be as efficient as sending a set of drawing commands over the network. This perceived (or real) inefficiency is what I think has a lot of people worried about this whole plan. Maybe I'm missing something here?
I understand that the current toolkits (GTK/QT) running under traditional X do this anyway, so given the current state of affairs we probably won't be losing anything.
However, I used X11 circa 1993-4 over a WAN across a good chunk of North America and it performed just fine for interactive editing and many other development tasks, despite the low bandwidth. I and many other students also used X Terminals over a 10 Mb/s Ethernet Lan to a shared workstation, and even interactive games didn't have any trouble. So I know the traditional X11 protocol (XDrawLine, etc) work perfectly well over the network.
So I'm wondering if the current toolkits evolved to do client side rendering because of problems with the X11 protocol and it's lack of progress when it stagnated under XFree86. The current toolkits wanted to use advanced drawing mechanisms (like OpenGL) so they basically bypassed X11 to do it and starting doing client side rendering.
Which brings me to this: OpenGL rendering commands are assumed to be asynchronous already (<http://www.opengl.org/wiki/Synchronization>). That being the case, would it make sense to send the OpenGL commands over the network instead of the buffer. All the commands up to the glFinish/glFlush are sent over to the "server", and then it "executes" them (possibly hardware accelerated) and then composites the result to the real screen. This doesn't really sound like what AIGLX was, or is it?
Does this make sense? What am I missing? I'm sure this has been thought of, I'm just trying to understand the reasoning behind not going that route.
Thanks,
OpenGL over the network? Posted Jun 23, 2012 4:16 UTC (Sat) by drag (subscriber, #31333) [Link]
> Does this make sense? What am I missing?
Well the only major thing that I can think of is that applications are not restricted to using OpenGL. The Wayland compositor uses EGL and theoretically can run on any system that supports the necessary extensions regardless of what driver stack they are using. However applications themselves can use whatever rendering method their authors deem most useful for themselves. In this way Wayland is simultaneously much simpler then X, but much more capable. Application developers can choose to use X or OpenGL or whatever other method is available. They just have to be able to write the output to a wayland buffer and be on their way.
The problem with X and remote desktop isn't really one of bandwidth, it's latency. X is a bit pathological compared to modern protocols on this account. Even if you are doing a relatively naive way of compressing images and shuttling them across the network it shouldn't be too dificult to do better then what X does.
OpenGL over the network? Posted Jun 23, 2012 9:03 UTC (Sat) by oak (subscriber, #2786) [Link]
Double buffering needed for flickerfree window updates was already mentioned above. X has an extension for this, but as toolkits cannot trust that to be always present they need to do double buffering themselves anyway.
Same problem applies to vector fonts / Xft2, antialiased line drawing with Cairo etc. As those are optional X extensions or still completely missing features, toolkit needs code for them. this applies especially to remote X usage as a lot of remote X usage has happened with proprietary X servers like Exceed on Windows which aren't very fast to support new extensions...
Doing rendering just on client side is the simplest solution for supporting all this.
There are also other advantages to client side rendering like having full control on what the rendering results look like and this CPU intensive activity being done on each client instead of all that being done on server side. Latter means that you see which client uses CPU and are able to control it more.
The funny thing here is that increasing DPI values probably will make antialiasing mostly redundant, only users with exceptional eye sight will be able to see any difference...
OpenGL over the network? Posted Jun 25, 2012 8:40 UTC (Mon) by renox (subscriber, #23785) [Link]
> Doing rendering just on client side is the simplest solution for supporting all this.
And X can support intelligent rendering on the client, render the glyphs on client, upload to server who cache them and reuse the glyphs for free, Wayland don't have this..
OpenGL over the network? Posted Jun 25, 2012 3:09 UTC (Mon) by rillian (subscriber, #11344) [Link] To me, this seems like it must very inefficient - no matter how smart you are, copying a big buffer of data can't be as efficient as sending a set of drawing commands over the network. This isn't necessarily true. The final display image can be larger or smaller than the data used to create it, depending on the complexity of the application. Updating a terminal window or a page of text? Smaller to send the font and the glyphs indicies. Updating a 3d rendering with several gigabytes of geometry and textures? Smaller to send the final output. This is why, as you say, 90s-era remote X11 works fine; the drawing uses high level commands which are more efficient to send than the bitmaps they create. It ought to work just as well for some gradients and anti-aliased text. But there's a limit. In any case, wayland doesn't address this issue at all. It assumes the client has rendered into a buffer, probably by running commands on the GPU, and the protocol is just about telling the compositor how to display that buffer and return events. So there's plenty of room to write both command-list and bitmap-flinging remote clients for wayland and integrate them into application toolkits, etc. The concern is that by removing the universal X11 network transparency and not replacing it with anything, those who use that feature will see it become even less useful. Just like -NXHost doesn't work on MacOS X.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 16:02 UTC (Sat) by alankila (subscriber, #47141) [Link]
I'm wondering if weston or such would be capable of getting color management going simultaneously.
In the future, I hope that we will use 64 bits per pixel bitmaps to draw UIs on, perhaps in some such color space as scRGB(16), and then let the compositor convert these for display on screen while aware of the color space of the target display device.
I hope Wayland/Weston does not assume that bitmaps are just 32-bit ARGB in the sRGB color space, as that could prove to be incredibly shortsighted.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 16:16 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]
Wayland supports YUV and other surface formats just fine.
St. Pierre: The Linux Graphics Stack Posted Jun 23, 2012 17:48 UTC (Sat) by alankila (subscriber, #47141) [Link]
Well, let's just say that I'm not reassured, unless color management is seriously thought about and there exists at least some plans to handle wide-gamut displays in a transparent way. I'd really prefer the simplicity of applications using scRGB(16) or other such linear-light colorspace with wide enough gamut to suffice in practice. My dream would be the default to not be 32-bit sRGB because so few people understand the nonlinearity of sRGB color space, and therefore getting artifact-free image scaling or font rendering continue to be the same uphill battle it has always been.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 17:25 UTC (Mon) by butlerm (subscriber, #13312) [Link]
>I'd really prefer the simplicity of applications using scRGB(16) or other such linear-light colorspace
Would an eight bit linear, standard gamut colorspace be enough of an improvement to be worth implementing? Or does that cause too much loss of color precision relative to 8 bit sRGB?
The main reason of course for supporting 8 bit is the extra resource requirements of 16 bit components, especially in a remoting protocol. Much more extensive and possibly lossy compression would have to be applied to keep up.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 17:34 UTC (Mon) by alankila (subscriber, #47141) [Link]
It will not have high enough quality, I'm sad to say.
If you keep the sRGB gamut, it takes about 12 bits of linear precision per component to still generate every 8-bit sRGB value. Of course, the problem is most severe at the darkest of shades, but it doesn't really change things too much to just go full 3x16 or 4x16 bits compared to some more limited solution.
For the remoting protocol, it is thinkable that the fidelity of the rendering would be reduced during the process, but the focus should be very strongly on getting good local behavior and treating the remote behavior as afterthought. It's probably not so hard as people think, and in any case we must not cripple local features for sake of preserving remoting, imho.
St. Pierre: The Linux Graphics Stack Posted Jun 27, 2012 4:37 UTC (Wed) by butlerm (subscriber, #13312) [Link]
16 bit linear ARGB sounds like it ought to become the new standard buffer format. I sure hope the GPU vendors adopt a different representation than standard scRGB(16) however, because storing reference black with a large positive bias is going to complicate processing enormously.
http://en.wikipedia.org/wiki/ScRGB
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 3:29 UTC (Mon) by rillian (subscriber, #11344) [Link]
I don't see any way the current api to attach a colour profile to a buffer, or query the display server for the colour profiles of the display devices. So the answer on color management remains "no".
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 9:18 UTC (Mon) by alankila (subscriber, #47141) [Link]
Yes... But it should totally be done. The competing display technologies do it, and it's a shame if we these basic features are not handled. And IMHO clients do not really *have* to know what the color space of the display system is -- they should just specify the buffer format (and implicitly the color space) they want to work with. The problem of translating the colors for the display subsystem can be handled transparently for them.
As I have mentioned before, it could be done through a 3D trilinear texture lookup, where source 3-component color could be mapped to destination RGB triplet, and this could be left for the hardware to do, rather than involve something expensive such as lcms (or even qcms) for each pixel. A shader for it is trivial to write, if shader is the way people want to do it.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 17:27 UTC (Mon) by Darxus (guest, #85281) [Link]
Post to the wayland mailing list. This has been brought up before, I don't know what the plans are.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 8:45 UTC (Mon) by njd27 (subscriber, #5770) [Link]
I think the bikeshed should be blue.
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 15:14 UTC (Mon) by welinder (guest, #4699) [Link]
> DRI 1 wasn’t really that good, so we threw it out and replaced it with DRI 2.
There's something scary about that statement coming from people who
St. Pierre: The Linux Graphics Stack Posted Jun 25, 2012 15:36 UTC (Mon) by raven667 (subscriber, #5198) [Link]
I think that's a little harsh, from what I can tell DRI1 was designed in 1998 around OpenGL 1.x, XFree86 3.x and kernel 2.2.x. DRI2 design was started in 2007 after a decade of experience and with much improved underlying technology, in the Kernel, in Xorg and in the design of GPUs.
http://dri.freedesktop.org/wiki/DriHistory
St. Pierre: The Linux Graphics Stack Posted Jun 29, 2012 8:41 UTC (Fri) by bronson (subscriber, #4806) [Link]
Compare a modern graphics chip to one from six years ago. Or DX9 to DX10 to DX11. Throwing out crusty old technology is normal for the entire graphics industry. Look at it as a sign of vitality.
|
|||||||||||||
