FOSDEM: The Wayland display server [LWN.net]

FOSDEM: The Wayland display server

Posted Feb 16, 2012 11:10 UTC (Thu) by dps (guest, #5725)
Parent article: FOSDEM: The Wayland display server

One of the nice things about X clients is that they work on remote displays with none of limitations you get with things like VNC (e.g. no cut and paste does not work between local and remote applications). If wayland breeds clients that can't do this then it is a giant leap backwards.

I *don't care* about flicker free movement of windows, video, or hardware accelerated graphics because I rarely do anything which would benefit. Sacrificing network transparency, which I *do* need, to get those is completely stupid.

That said I am writing software for very expensive server hardware.

to post comments

FOSDEM: The Wayland display server

Posted Feb 16, 2012 12:05 UTC (Thu) by angdraug (subscriber, #7487) [Link] (18 responses)

This was already discussed to the death and beyond in the previous post about Wayland on LWN, please lets not have another 400-comment thread about this here.

FOSDEM: The Wayland display server

Posted Feb 16, 2012 14:03 UTC (Thu) by faramir (subscriber, #2327) [Link] (1 responses)

Sorry can't let it go. The article quotes Wayland's primary author as follows:

"While supporting remote displays is not a priority now for Wayland, nothing in Wayland's architecture makes this impossible."

The Wayland FAQ is more detailed and to me makes it appear problematic.

http://wayland.freedesktop.org/faq.html#heading_toc_j_8

"Is Wayland network transparent / does it support remote rendering?

No, that is outside the scope of Wayland. To support remote rendering you need to define a rendering API, which is something I've been very careful to avoid doing. The reason Wayland is so simple and feasible at all is that I'm sidestepping this big task and pushing it to the clients.

*** It's an interesting challenge, a very big task and it's hard to get right, but essentially orthogonal to what Wayland tries to achieve. ***

...."

I'm also concerned about all of these toolkits that are being ported to Wayland. Will applications that are linked against them still talk to X Servers? Or is this something that a developer will have to decide when they build their application? I understand that this is outside the scope of Wayland, but am wondering what decisions are being made. Will I have to have two copies of simple apps that I happen to want to sometimes run remotely? (At least until the "very big task" of adding network transparency to Wayland is complete.)

FOSDEM: The Wayland display server

Posted Feb 16, 2012 14:54 UTC (Thu) by khim (subscriber, #9252) [Link]

I'm also concerned about all of these toolkits that are being ported to Wayland. Will applications that are linked against them still talk to X Servers? Or is this something that a developer will have to decide when they build their application?

This depends on the toolkit, obviously, but GTK+ and Qt support plugable backends. I think GTK+ 3.0 was one of the last components which were needed before something like Wayland become feasible.

Note that even if toolkit itself supports switching application can still be tied to one implementation or the other.

FOSDEM: The Wayland display server

Posted Feb 16, 2012 15:24 UTC (Thu) by bkw1a (guest, #4101) [Link] (15 responses)

The fact that this drew 400 comments is a clear indication that it's important to a lot of people. If it's really not a big deal to add network transparency to Wayland, maybe it should be done sooner rather than later, just to reduce people's level of anxiety.

FOSDEM: The Wayland display server

Posted Feb 16, 2012 15:50 UTC (Thu) by angdraug (subscriber, #7487) [Link] (14 responses)

I'm not saying it's not important, I just don't see any value in repeating the same discussion within the space of a few days.

I also don't think that the number of comments or level of anxiety is a good substitute for a sound technical argument.

Finally, it is a big deal to add network transparency to Wayland: it is architecturally feasible, but it is still a lot of work. It just doesn't make sense to divert the scarce resources to this work until the primary usage scenario is in good shape.

FOSDEM: The Wayland display server

Posted Feb 16, 2012 18:16 UTC (Thu) by JoeBuck (subscriber, #2330) [Link] (11 responses)

Precisely because so many people care about network transparency, I'm confident that some of those people will actively contribute to efforts to support it. This is free software after all, it's not like we're helpless because Steve Jobs took out our favorite feature.

Perhaps some highly motivated people can put together a design for a kick-ass way to support remote access with Wayland that avoids the problems with the X implementation (too many round trips, too much bandwidth required). A remote access protocol that is aware of structure should be able to achieve much better performance than VNC.

FOSDEM: The Wayland display server

Posted Feb 17, 2012 10:54 UTC (Fri) by alankila (guest, #47141) [Link] (10 responses)

I said this before and will say again: toolkit-level network transparency can be experimented with today, whether X11 or Wayland is the display server, and it should yield superior performance, and have some other goodies: for instance, it could render the GUI according to user's locally defined GTK+ theme rather than whatever pixels the server wants to push (because server doesn't push pixels, it will push statements like 'add a button with these attributes inside that container, please').

We should *absolutely* start doing this work, and we might prefer it over X11 even if Wayland never actually lands on our desktops.

FOSDEM: The Wayland display server

Posted Feb 17, 2012 13:22 UTC (Fri) by Tet (subscriber, #5433) [Link] (3 responses)

toolkit-level network transparency can be experimented with today

But why would you want to? The toolkit is absolutely the wrong place to be doing this. As I write this, I have applications open using at least 4 different toolkits. Implementing network transparency in each of them such that it behaves the same in all of my current applications is simply not going to happen. If you believe otherwise, I have a bridge I'd like to sell you...

FOSDEM: The Wayland display server

Posted Feb 17, 2012 14:14 UTC (Fri) by alankila (guest, #47141) [Link] (2 responses)

Maybe this would get people to stop using so many toolkits...

In any case, in Wayland's architecture -- which does not define rendering at all -- there is no other obvious way to do the thing except by repeated bitmap snapshots of the window contents followed by costly analysis and encoding of the difference. It will suck.

FOSDEM: The Wayland display server

Posted Feb 23, 2012 19:31 UTC (Thu) by wmf (guest, #33791) [Link] (1 responses)

I think it's too early to declare that compressed bitmaps will suck. For example, the Sun Labs SLIM project found that a simple VNC-like protocol used less bandwidth than X11.

FOSDEM: The Wayland display server

Posted Mar 1, 2012 13:07 UTC (Thu) by nix (subscriber, #2304) [Link]

In what use case, though? Easy stuff like a canvas with stuff being drawn on it, or hard-but-common stuff like a scrolling xterm window filled with text?

FOSDEM: The Wayland display server

Posted Feb 18, 2012 2:24 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

Toolkit-level transparency is actually even used (somewhat). There's RSWT: http://rswt.sourceforge.net/screenshots.html and several similar projects (like Remote SWING).

Making a remote GTK or QT UI is feasible and should not even be hard. AND since in that case only very little data is going to be transmitted, it can actually be useful.

FOSDEM: The Wayland display server

Posted Feb 18, 2012 21:04 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

I would think that the toolkits would also be able to use the local D-Bus sessions, ssh-agent sockets, etc. instead of also having to forward those in addition to the X session socket (or just leaving it as-is and always hooking up to the remote one).

You could have automatic gvfs and KIO slaves made for wayland-local:/ and wayland-remote:/ protocols so that the local and the remote filesystems could be accessed. There's a lot more than just pushing pixels across a pipe to get the full experience of running a remote application as if it were local.

FOSDEM: The Wayland display server

Posted Feb 20, 2012 22:11 UTC (Mon) by HelloWorld (guest, #56129) [Link] (3 responses)

> I said this before and will say again: toolkit-level network transparency can be experimented with today, whether X11 or Wayland is the display server, and it should yield superior performance, and have some other goodies: for instance, it could render the GUI according to user's locally defined GTK+ theme rather than whatever pixels the server wants to push (because server doesn't push pixels, it will push statements like 'add a button with these attributes inside that container, please').
That sounds nice in theory, but I doubt it'll work. For example, what if the client uses GTK+ version n, and the server only has version n-1 available?
Also, you'd need to run one server for Qt, one for GTK+, one for EFL etc. etc., that's going to be a nightmare.
What I'd like to see is a rendering server that allows clients to upload little programs that know how to, say, draw a button, and then in order to draw a button, they only have to tell the server to run that program they've uploaded before using such and such parameters. I think DisplayPostscript worked like that. That would at least get rid of the need to run multiple servers for the toolkits, as they could more or less define their own specialized protocol.
However, I'm not sure that'll work either. It would probably require rather substantial changes in the toolkits, and it would probably be hard to port existing theme engines etc. to such a new paradigm, at least if one wants to exploit this kind of technology to a significant degree. Also, we already have similar technology: JavaScript and Canvas/WebGL. GTK+ already features the Broadway backend which renders into a browser window, though I'm not sure how much work is actually done on the client or on the server side.
So clearly, this is a hard problem to solve.

FOSDEM: The Wayland display server

Posted Feb 21, 2012 11:52 UTC (Tue) by alankila (guest, #47141) [Link] (2 responses)

Yes, it is a hard problem to solve, especially if you want to solve it for all software there is... I'd personally settle for far less: it's enough to me that *one* reasonably good toolkit does it, especially if it's GTK+ or Qt: most software already exists for both of these toolkits, so you just have one more reason to avoid the worse one.

FOSDEM: The Wayland display server

Posted Feb 21, 2012 13:23 UTC (Tue) by HelloWorld (guest, #56129) [Link] (1 responses)

I disagree with this. I'm a KDE user, yet I also use quite a few GTK+ applications because their Qt alternatives either don't exist or suck, for example wireshark, emacs and leksah. Considering this, a remote GUI that can't handle at least GTK+ and Qt would be next to useless in my opinion.

Also, you didn't explain how to solve the issue of differing GTK+ versions between the client and the server.

FOSDEM: The Wayland display server

Posted Mar 1, 2012 13:10 UTC (Thu) by nix (subscriber, #2304) [Link]

For that matter, I still run one single Xt application: Emacs. That's because Gtk dumps core if its X session goes away, making the emacs server mostly useless: the bug for this has been open for years and appears unfixable. Now maybe this problem will vanish with Wayland (as it relates to Gtk keeping stuff on the X server but not keeping track of it so it can resend it if the X connection dies)... but maybe not. Demonstrably the Gtk hackers do not care about use cases like remotely running an application which maps multiple windows, then unmapping all the windows so you can suspend the machine on which the display is done.

Because, after all, saving power by suspending your machine when not in use while leaving your processes running on another machine so they can keep doing background jobs is a rare use case, unless you jam the word 'cloud' in there, whereupon it suddenly becomes immensely important. Or something.

FOSDEM: The Wayland display server

Posted Feb 22, 2012 7:44 UTC (Wed) by mina86 (guest, #68442) [Link] (1 responses)

> Finally, it is a big deal to add network transparency to Wayland: it is architecturally feasible, but it is still a lot of work.

If Wayland is designed with assumption that everything is done on client side and that client and rendering runs in the same process, they whole system is bound to be created in such a way that creating network transparency in a way that is something different then transferring (probably compressed) bitmaps (eg. VNC) is almost impossible.

FOSDEM: The Wayland display server

Posted Feb 22, 2012 16:26 UTC (Wed) by dgm (subscriber, #49227) [Link]

Not true. The very fact that you can actually run X11 on top of Wayland is enough counter-proof. In Wayland the client is responsible for rendering, but the logic governing that rendering could be wherever you want: in the same process or in a server on the other side of the World.

To put things in perspective, that's a conceptual graphic stack.

(1)logic -> (2)drawing -> (3)window -> (4)desktop -> (5)display

Here "logic" is your program code. "Drawing" is the X11 and toolkit level. "Window" is a fully painted window, maybe including decorations. "Desktop" is the full desktop of overlapped windows, and display is the actual framebuffer and screen.

network between (1) and (2) is the HTML model. The logic is in the server, but the drawing code (the browser + javascript) is in the client.
network between (2) and (3) is the X11 model. The drawing code is in the server, but the window is in the client.
network between (3) and (4) is seamlessRDP. The Window is in the server, but the full desktop is not.
network between (4) and (5) is the VNC model. Everything is in the server, the client is just a remote screen.

Wayland starts at 3. It takes fully painted windows, composites them and takes them to the display. You could add network transparency _inside_ of Wayland with window buffer forwarding (seamlessRDP style). But you could also add network transparency outside of Wayland, for example VNC style using a virtual display. Or at the drawing level using a networked drawing library (X11). Or at the logic level with an HTML application. Of all those possibilities, only a seamlessRDP style protocol is not developed yet.

FOSDEM: The Wayland display server

Posted Feb 16, 2012 17:32 UTC (Thu) by iabervon (subscriber, #722) [Link] (2 responses)

I'm not seeing where in the architecture Wayland supports cut and paste between local applications (or, in general, any sort of interaction between clients). So remote applications won't be second-class citizens in that sense, because it isn't supported at all. I suspect that there will be some mechanism outside of Wayland for it, and it will hopefully be something ssh/sshd will quickly learn to proxy. And, for that matter, it would be great if that covered other inter-client communication that X doesn't presently carry, like opening URLs in your web browser and libnotify.

FOSDEM: The Wayland display server

Posted Feb 16, 2012 20:10 UTC (Thu) by angdraug (subscriber, #7487) [Link]

I hope there will be a datagram based protocol, something that can be wrapped in a UDP-based OpenVPN or IPsec tunnel instead of SSH, for better latency. Putty in an RDP session is as much pain as xterm over VNC.

FOSDEM: The Wayland display server

Posted Feb 17, 2012 7:22 UTC (Fri) by JohnLenz (guest, #42089) [Link]

Actually, that isn't correct. The protocol includes direct support for copy/paste and drag/drop. See this old thread for the rationale to add it directly to the protocol. You can notice the copy/paste stuff in the current protocol description.