LWN: Comments on "Announcing printerd"

You betray your ignorance of wayland

aquasync — Wed, 30 May 2012 05:47:26 +0000

I would expect so, using the GDK_BACKEND environment variable (see the gtk broadway demos).

forwarding unix-domain sockets over ssh [was: Announcing printerd]

dkg — Wed, 30 May 2012 01:35:56 +0000

Yes, you can certainly use socat with ssh to forward UNIX-domain sockets.

You betray your ignorance of wayland

nix — Tue, 29 May 2012 21:53:10 +0000

Yeah. If Gtk does that too, most objections to Wayland basically fall away: nobody's likely to implement Wayland-only programs (just as people don't write against raw Xlib these days), and if the toolkits can switch based on an env var or something, the user can choose with whatever granularity is desired between insane blazing performance and remotability (which means I can turn remotability on and forget about Wayland completely).

You betray your ignorance of wayland

butlerm — Tue, 29 May 2012 07:51:31 +0000

Thanks, that is excellent news. Perhaps a little more work than running everything through a generalized version of the XCB API, but accomplishing the same thing nonetheless, and probably more flexible too.

You betray your ignorance of wayland

Jonno — Mon, 28 May 2012 17:40:34 +0000

I don't know about GTK+, but Qt 4.8 and 5.0 autodetects backend at runtime, which can be overridden by a command line argument to the application (i.e. if you have both Wayland and X11 running).

NB: For Qt 4.8 you need to explicitly configure Qt with -qpa to get the new modular backends, the default is X11 only. In Qt 5.0 the hard-coded X11 port will be gone and replaced with a modular xcb backend.

You betray your ignorance of wayland

raven667 — Mon, 28 May 2012 16:19:05 +0000

I thought the way this was being implemented in the toolkits was to check for environment variables to know which output to use.

You betray your ignorance of wayland

butlerm — Mon, 28 May 2012 15:59:15 +0000

>The existing robust X backends aren't just going to evaporate, the toolkits are already multi-platform and Wayland is just an additional backend.

My question here is basically can the toolkit switch dynamically so that the same application binary be run in both X mode or Wayland mode, depending on the runtime environment?

If so, that would be outstanding, if not that could put a major impediment in the deployment of such applications until some form of remote operation is working. I can't see a server distribution standardizing on applications that can't be accessed remotely.

Announcing printerd

nix — Mon, 28 May 2012 13:01:57 +0000

Probably. After all, socat can do anything. Downside: reading the manual requires dark rites first to prevent your brain dribbling out of your ears. It's the Emacs of netcats.

You betray your ignorance of wayland

raven667 — Mon, 28 May 2012 06:43:34 +0000

> RDP is based on T.128

I'm not sure if that's the chicken or the egg, these protocols (RDP, ICA) have been deployed for more than 20 years and I'm not sure that the standards you references aren't just ex post facto documentation of existing practice.

> the usual set of rendering operations, font and image caching, and so on

One more thing to point out is that all the protocols I listed are add-ons to an existing system (mostly MS Windows) that isn't designed around remoting applications and so has to integrate with an infrastructure designed around local display, the same kind of problems that would need to be solved to remote native Wayland applications.

The methods for local display and remote display have little relation to one another and I think experience has shown that this is a good arrangement, trying to do both with the same method leads to a design that is poor for both cases.

Does any of that sound similar to your understanding of the situation? 8-)

You betray your ignorance of wayland

raven667 — Mon, 28 May 2012 06:33:42 +0000

> Presumably, the use of direct-to-Wayland backends with toolkits like GTK and QT will normally make things worse (if not impossible) for remote operation.

The existing robust X backends aren't just going to evaporate, the toolkits are already multi-platform and Wayland is just an additional backend.

> So are the advantages of Wayland with X layered on top substantial enough that it is a major improvement over the status quo?

Yes, it takes the hardware handling out of X and leaves it with just the protocol to support existing apps. The hardware handling can be much better done outside of the X framework than within it. X can be much simpler and more robust if it isn't trying to handle the underlying hardware.

> Or do the real advantages of Wayland only come when you eliminate the X layer in-between?

There is probably sufficient advantage even if the only native Wayland client was the X server although in practice toolkits are going to have a direct Wayland option and eliminate the middleman for local display (only).

> Couldn't you just have libXCB or whatever have multiple backends, one a stub for remote operation using the X protocol, and the other an in-process rasterizer for use with Wayland, etc?

I may be talking above my knowledge level but my guess is that X apps and toolkits just aren't architected in a way where that makes any sense. The people who are designing and implementing Wayland are also the ones who designed and maintain X so my guess is that they are aware of the different architectural options and are picking the best ones.

You betray your ignorance of wayland

butlerm — Mon, 28 May 2012 05:52:04 +0000

RDP is based on T.128, which has the usual set of rendering operations, font and image caching, and so on. I understand that ICA is similar, and so is SPICE. Of course actual implementations may rely more on bitmap updates then the supported rendering commands these days for relatively obvious reasons.

You betray your ignorance of wayland

raven667 — Sun, 27 May 2012 21:12:10 +0000

Option 2 is probably the better architecture, every client will get it for free and it is the technique that has the most research and work behind it. As far as I know the main protocols (ICA, RDP, PCoIP, SPICE, VNC) are all designed this way at the core and are all using similar optimization techniques and research to reduce bandwidth usage and reduce lag. What kind of rendering protocol is the OnLive service using?

You betray your ignorance of wayland

Jonno — Sun, 27 May 2012 10:25:44 +0000

With Wayland (the protocol) there are two possible ways to do remote operation:

Option 1: Remote rendering *on top of* Wayland. In this scenario clients (applications) don't do rendering themselves, but sends rendering commands to the display server, which render the windows and ask the Wayland compositor to display them. This will ofcourse require changes to the clients, or to the toolkit they use. You can use the standard compositor (e.g. Weston) on the display server, but will need a rendering server and protocol as well. One existing solution is to use xserver and X11, but developing a better protocol is a possible future improvement.

Option 2: Remote composition *below* Wayland. In this scenario clients render the windows themselves and ask the local compositor to display them. The local compositor, however, does not, but instead compress the windows and sends them over the network to the display server, which in turn asks its Wayland compositor to display them. This will work with all Wayland clients without modification. You can use the standard compositor (e.g. Weston) on the display server, but will need a proxy compositor and client to forward the windows over the network. This is not yet implemented in production-ready code (though there are abandoned proof-of-concept code for older Wayland versions), but is expected to be the default once done.

Option 2 is often called VNC-like because you send pre-rendered images over the network, but unlike actual VNC, you will get rootless windows that integrate just as well as remote X11 windows, so a better description would be Xpra-like. For most (but not all) applications option 2 will require more bandwidth than option 1, but option 2 will require less roundtrips and are thus not as sensitive to lag (a.k.a. poor ping times).

User of dial-up connections and users on over-saturated local networks might want to deploy option 1 (i.e. no change from today), but most use cases will be better served by option 2 (i.e. better performance than today).

You betray your ignorance of wayland

butlerm — Sat, 26 May 2012 23:51:02 +0000

>This doesn't mean that remote rendering won't be possible with Wayland, it just means that you will have to put a remote rendering server on top of Wayland. One such server could be the X.org server, but other options include an RDP server, a VNC server or somebody could even invent their own new remote rendering model

This is where I get confused. Presumably, the use of direct-to-Wayland backends with toolkits like GTK and QT will normally make things worse (if not impossible) for remote operation. So are the advantages of Wayland with X layered on top substantial enough that it is a major improvement over the status quo? Or do the real advantages of Wayland only come when you eliminate the X layer in-between?

Furthermore, if Wayland does the compositing and the input arbitration, is it really necessary for X to run out of process in this case? Couldn't you just have libXCB or whatever have multiple backends, one a stub for remote operation using the X protocol, and the other an in-process rasterizer for use with Wayland, etc?

Because CUPS finally 'just works'

butlerm — Sat, 26 May 2012 23:23:16 +0000

>When my printer job does not succeed (PostScript error, paper out, pushing the Job Cancel button on the printer, ...) then "lpq" on my machine says "Printer is stopped", and indeed no progress will be made on any print job for that printer. So how do I restart the printer?

Whatever the rationale is, the default for this on CUPS could hardly be more unfriendly. This sort of thing makes it look like a full employment program for system administrators. No wonder people want to replace it.

Announcing printerd

jond — Sat, 26 May 2012 14:33:24 +0000

I often do that because the GNOME print wrappers around CUPS aren't very good. Actually I don't use https.

But what's the better alternative to Wayland?

sdalley — Fri, 25 May 2012 22:25:53 +0000

> let's not pretend that the existence of Wayland is a good thing..

There seems to be some memo that I didn't get. _Why_ is the existence of Wayland a _bad_ thing? What else would be better? It is a long overdue refactoring of the display/rendering/windowing plumbing that is careful not to reinvent new wheels but replace old and wobbly ones. In the medium term it will allow X to drop a load of old cruft.

This will surely end up making the overstretched X maintainers' lives easier, and more fulfilling than bodging improvements that are better and more simply done elsewhere.

It seems to be well supported by Red Hat and hopefully Canonical will soon be getting actively involved too.

As for remote-X issues, I certainly don't disagree there's problems there.
Using Linux desktops over network X clients has been getting steadily worse over recent years, not because X is getting worse but because it is being asked to do more and more. GLib/cairo-based software expects textures and gradients where previously it was just pixels. SunRay thin clients got round to adding the X RENDER extension a couple of years ago, making the performance of GLib-based software on them merely sucky instead of snailishly unusable.

These problems, however, are not within Wayland's scope at present. Its design carefully leaves a clear field in which to fix them. And so long as workarounds like Xvnc and Xpra continue to work, we should at least not go backwards.

What's not to like? It seems to me that Wayland deserves our support and help during the inevitable teething issues rather than criticism.

Strong words

renox — Fri, 25 May 2012 18:56:43 +0000

> Have you got some constructive suggestions to improve the situation?

Not really: the reason why X is not in a good state is due to chronic lack of manpower ( http://www.phoronix.com/scan.php?page=news_item&px=MT... )..

Wayland will "help" by doing less, OK, but let's not pretend that the existence of Wayland is a good thing..

Strong words

dlang — Fri, 25 May 2012 18:30:33 +0000

the place that X is weakest is the place that Wayland is punting on. that's hardly the "don't interrupt the person doing it" situation.

Strong words

sdalley — Fri, 25 May 2012 17:16:58 +0000

Sorry, I didn't mean to upset anyone. But I tend to have more confidence in the answers of those who designed the architecture, went to some trouble to document it, who've rolled their sleeves up and are actually doing the work, than in people who make snide potshots from the sidelines.

We all wish that the Linux desktop experience was better today. But what are your technical objections to Wayland? You admit that the current state of X is non-optimal. Have you got some constructive suggestions to improve the situation?

"The one who says something cannot be done should never interrupt the one who is doing it."

Strong words

renox — Fri, 25 May 2012 14:28:07 +0000

And what you demonstrate is that you don't understand that 'project XYZ FAQ' is usually biased towards project XYZ..

Obviously Wayland devs would say that that Wayland won't create issues for using remote display in a WAN, in practice we will see: it's very possible that this will be a mess..

In a way, it *is* already a bit of a mess: NX isn't integrated in X as it should be IMHO.

You betray your ignorance of wayland

sdalley — Fri, 25 May 2012 11:55:36 +0000

Ho, ho, very amusing. But you and the GP demonstrate that the Wayland people haven't been able to teach you anything at all. Remote display of an entire desktop (as opposed to an X window) is an orthogonal problem. See the Wayland FAQ:

Is Wayland network transparent / does it support remote rendering?
No, that is outside the scope of Wayland. To support remote rendering you need to define a rendering API, which is something I've been very careful to avoid doing. The reason Wayland is so simple and feasible at all is that I'm sidestepping this big task and pushing it to the clients. It's an interesting challenge, a very big task and it's hard to get right, but essentially orthogonal to what Wayland tries to achieve.

This doesn't mean that remote rendering won't be possible with Wayland, it just means that you will have to put a remote rendering server on top of Wayland. One such server could be the X.org server, but other options include an RDP server, a VNC server or somebody could even invent their own new remote rendering model. Which is a feature when you think about it; layering X.org on top of Wayland has very little overhead, but the other types of remote rendering servers no longer requires X.org, and experimenting with new protocols is easier.

It is also possible to put a remoting protocol into a wayland compositor, either a standalone remoting compositor or as a part of a full desktop compositor. This will let us forward native Wayland applications. The standalone compositor could let you log into a server and run an application back on your desktop. Building the forwarding into the desktop compositor could let you export or share a window on the fly with a remote wayland compositor, for example, a friend's desktop.

As far as existing remote X functionality is concerned, look at http://wayland.freedesktop.org/architecture.html , and then at http://wayland.freedesktop.org/xserver.html . You will see from this that the Xserver merely becomes another Wayland client. The intention is that nothing that X was able to do before (e.g. remote windows over the network) is going to be lost, because remote X clients can continue to connect to the X server just as they always have.

Announcing printerd

dgm — Fri, 25 May 2012 10:35:32 +0000

Wrong. You bring your monitor with you down to the Xerox machine and copy _that_. I hate when people don't pay attention.

Now seriously: that's exactly what you have been doing for a few years now unless you limit yourself to Motif applications, because all popular toolkits draw on the client side and then send the pixmaps to the X display via XRender these days. Or at least that's what they told to me.

Announcing printerd

quotemstr — Fri, 25 May 2012 03:45:02 +0000

You don't need network printing support. You can just print your document locally, walk over to the network printer, scan it there, then print another copy. It's _BETTER_ than old-fashioned network printing.

Or at least that's what the Wayland people taught me.

Announcing printerd

butlerm — Thu, 24 May 2012 18:35:24 +0000

> the only plus is that it can start printing when it completed the image of the first page.

If there isn't any serious way to generate and stream a PDF document one page at a time, PDF would seem to be a uniquely unsuitable format for the printing of large documents not already composed and linearized in PDF format in any kind of a hurry. Perhaps SVG Print would be a better long term alternative for that reason alone.

Announcing printerd

jsanders — Thu, 24 May 2012 18:25:26 +0000

Oh, it turns out in my ignorance, PDF is much more of a subset than I thought... I thought it still had control structures, but they are not there.

Announcing printerd

jsanders — Thu, 24 May 2012 18:02:08 +0000

I think you are mistaken if you think PDF is not a programming language. PDF is just a packaged-up subset of PostScript, with some enhancements and conventions for adding metadata. It's true that PDF doesn't have the full freedom PostScript has (like being constrained to one page at a time), but it really is a programming language.

Announcing printerd

njd27 — Thu, 24 May 2012 15:33:15 +0000

I have to say, I remote print over the network all the time, and I find writing a new print spooler without building network support in from the beginning to be a backward step. This is just like those wayland people all over again.

Announcing printerd

hppnq — Thu, 24 May 2012 09:31:01 +0000

Perhaps one could use socat.

Announcing printerd

pkern — Thu, 24 May 2012 09:00:49 +0000

You really want to drive network-attached printers with rasterized input as well. It has the computing power of a toaster, the only plus is that it can start printing when it completed the image of the first page.

PDF can have multiple layers and hence may overload a rasterizing printer easily. Even PCL XL is already too complex and certain outputs will cause the printer to take a walk.

The only part that worries me about printers that really only take raster input is the fact that you then need to bother with uploading their firmware and the newest and greatest proprietary inventions of rasterization on-the-wire formats to reverse engineer.

Announcing printerd

butlerm — Thu, 24 May 2012 08:30:12 +0000

>A network-attached printer ought to be able to perform its own rasterization, to conserve I/O.

This is a nice benefit, but the problem is that in most cases it is a program of planned obsolescence for otherwise perfectly adequate printer hardware. Short of mid-range printer manufacturers adopting a standard for rasterizer plugin cards, host (or print server) based rasterization ought to be the most effective way to keep a decent printer from not so gradually turning into an expensive paperweight.

As an example, I have a ~$1K laser printer from a name brand manufacturer that is less than five years old that is in the habit of taking more than a minute to print single pages from innocent looking PDF files. Certainly any well written host based rasterizer could run circles around that.

>Modern OpenGL (with shaders rather than fixed-function pipeline stages) essentially _is_ a Turing-complete programming language.

OpenGL serialization was a poor choice for an example. Something more like a efficient serialization of the PDF rendering model was what I had in mind. A serialization of OpenVG might work nicely, but it appears to be a little too simple.

Architecture astronauts take over

jku — Thu, 24 May 2012 07:05:44 +0000

"sounds like a project that is started on horribly wrong basis" -- this is how he starts. Maybe he intended to say what you said ("announcement doesn't give any indication of what the project is for"), but I'd say there is room for misunderstanding there... There's a world of difference with "there's not enough information to evaluate this project" and "this project started on the wrong basis".

It's true that the announcement should have come with a link to http://cyberelk.net/tim/2012/05/23/some-benefits-of-print... but lets not make that omission a larger issue than it is.

Announcing printerd

nybble41 — Thu, 24 May 2012 02:53:35 +0000

I'm talking about vertex and fragment shaders, in OpenGL terms, not generating text via shader programs. The scene graph would include the text as references to glyphs, and any required bitmaps in the form of textures. The shaders would be handling transparency, gradients, smooth scaling, and other visual effects, just as with rendering to a display.

Announcing printerd

nix — Thu, 24 May 2012 00:34:36 +0000

Interesting! I'll have a look at that...

Announcing printerd

nix — Thu, 24 May 2012 00:28:14 +0000

You can't forward AF_UNIX sockets with OpenSSH generally :( like I said, doing this properly requires changes to OpenSSH too.

Announcing printerd

nix — Thu, 24 May 2012 00:26:21 +0000

Shaders?! You do realise that most printers are used mostly to print text, or bitmap representations of text, and that rendering text via shaders is insanely resource-expensive?

Like it or not, there's going to be either a bitmap or a vector language (or, more likely, both) in any plausible printer for a very long time to come.

Architecture astronauts take over

jjs — Wed, 23 May 2012 23:02:57 +0000

I read his articles, and am still uncertain of the benefits. CUPS talks fine to remote printers. He admits CUPS integrates with polkit. Right now I tell my app to print, answer the various questions (# of copies, printer, etc), and off it goes. I go back to my app,and sometime later the printer prints my work (asynchronous from my viewpoint).

So other than adding another layer between the application and CUPS, what does printerd do?

Because CUPS finally 'just works'

jjs — Wed, 23 May 2012 22:56:32 +0000

lp* is still on my debian testing system.

Architecture astronauts take over

cmccabe — Wed, 23 May 2012 22:41:28 +0000

To be honest, I really don't care about asynchronous printing. How can I say this in the most tactful way-- I care more about dusting my ceilings than I do about the fact that applications need to spawn an extra thread to print.

My printer is a USB printer. It requires a printer driver to work. A driver which, incidentally has not been updated or fixed in years, and is not officially supported by the distro. Luckily I know enough about the plumbing to know how I can get this thing to work. (Should "work" be in quotes?)

At work, there is an IPP printer which I mostly use. The time I spent setting it up in CUPS was about 5 minutes, including printing test pages and all that.

> Running in the session has some advantages besides just needing less code.
> It also allows the spooler to do things like authenticate to the cloud
> service in a normal way, as its part of the user desktop session, rather
> than some system service.

Again, I just don't care. I've never used a multi-seat Linux machine (and I've had 5 jobs where I used Linux, including one where we used it in the cloud.) I also don't see what fundamentally prevents CUPS from doing the same thing-- i.e., accepting or rejecting jobs based on the user id that submitted them.

So far, printerd has not proposed to solve any of the problems I have ever had. Of course, I'm just one user-- perhaps others will have different needs.

Announcing printerd

nybble41 — Wed, 23 May 2012 20:30:03 +0000

> As a consequence of this common architecture, a Turing-complete programming language ... is massive overkill for contemporary printer interface. ... A high performance page rendering interface should be based on ... a serialized version of OpenGL.

Modern OpenGL (with shaders rather than fixed-function pipeline stages) essentially _is_ a Turing-complete programming language. For similar reasons, I don't think non-Turing-completeness is a reasonable requirement for any rendering API, and for printing I'd actually prefer something like a scene graph with attached shader programs. Naturally, as with rendering for the display, the shaders would be subject to various resource limitations, including a limit on runtime. It should be possible to rasterize the print job via commonly-available embedded, OpenGL ES-capable GPUs.

> Or why we shouldn't prepare for a world where nearly all page rendering is handled on the host system rather than by a ridiculously underpowered, buggy, and resource impaired piece of firmware on the printer?

A network-attached printer ought to be able to perform its own rasterization, to conserve I/O. For printers designed to be connected only to a host PC, however, I'd just as soon standardize on a simple raster-based printing protocol and do the heavy lifting on the host--ideally with GPU acceleration.