Two sessions at the Ottawa Linux Symposium hosted by Keith Packard and Jim
Gettys combined to give a good overview
of where the X window system is going. This article is an attempt to pull
together the material covered in those talks. X, which, in Jim's words,
"just sat there for years with
nothing interesting happening," is now the focus of a flurry of activity.
Expect to see interesting things happening over the next year or two -
especially if a few more developers can be brought into the fold.
Keith Packard's talk had to do with separating the X server from the
hardware. X currently does all kinds of interesting things, from direct
programming of video modes through to remapping the PCI bus (to get around
interesting BIOS behavior), scary DMA operations from user mode, direct
interpretation of mouse events, and no end of user-space busy-waiting.
There are all kinds of unfortunate side effects from this behavior;
remapping the PCI bus and attempting DMA from user mode can bring down the
whole system without much trouble. X's busy-waiting behavior leads, among
other things, to poor interaction with the scheduler, which sees X as a CPU
hog process whose priority should be lowered. Things would improve greatly
if X did not get its hands quite so dirty with the hardware.
Interestingly, Keith blames himself for much of the current situation.
Very early X11 implementations dealt with the operating system for their
frame buffer access. Then Sun came out with a closed video adaptor which
required some user-space tweaking; Keith duly hacked this up. More closed
hardware came out, making the problem worse. Things led to the current
situation, where X knows all about the hardware, deals with it at a very
low level, and occasionally lets things go wrong in spectacular ways.
Things are, says Keith, starting to get better. The direct rendering
infrastructure (DRI) is "a glimmer of sanity"; it returns
DMA and interrupt handling to the kernel. The kernel frame buffer
subsystem has helped to move hardware detection and mode selection back
into the kernel. And /dev/input has gotten X out of the business
of directly interpreting mouse sequences and trying to figure out which
protocol is in use; the input subsystem also allows marking input events
with proper, accurate timestamps, which is important to many applications.
The input subsystem is still not perfect, however, especially on multi-head
systems: there is no easy way to associate input devices with specific
The process of separating X from the hardware needs to continue, however.
X is still often in the business of configuring video modes, which is a
complex and unforgiving task. "Everybody just hopes that X gets it right."
This task should be moved out of X, and into some appropriate combination
of kernel and user-space code.
Keith's vision goes beyond mode selection, though; he would eventually like
to see the X server using GL to speak with the hardware. Turning X into a
GL client would make proper 3D support easier, would further separate X
from the hardware, and would give hardware vendors a single set of drivers
A huge remaining issue with the hardware side of X is hotplug monitors.
The assumption that the available screens will never change is wired very
deeply into the X server, and will not be easy to root out. But this work
must be done; people are increasingly wanting to, for example, plug their
laptop into different projectors from one day to the next, and it would be nice if it
Jim Gettys's talk covered just about every aspect of the window system -
and most of those are currently in flux. The fork in X development, which
saw almost all of the XFree86 developers moving over to X.org, has
revitalized development in a number of ways. Many longstanding problems
are being addressed, and lots of new ideas are being kicked around.
Jim started with a list of current issues with X:
- The X font architecture, as originally designed, was terminally
- The internal 2D graphics engine was poor; the plan from the beginning
had been to replace it quickly. Once again, we see how "temporary"
code can stay with us for decades.
- For various reasons, screen magnifiers and other accessibility aids
are very hard to implement.
- X needs better "eye candy" support: translucent graphics, drop
shadows, etc. Competing window systems (OS X, Longhorn) are
raising the bar in this area, and X needs to stay up with them - if
- Three-dimensional graphics are increasingly commonplace, and must be
- Shared use of X (shared projector walls were an example that came up
repeatedly) is poorly supported currently.
Anybody who has worked with X in any depth knows that its font mechanism
has not aged well. What is happening now is that fonts are, increasingly,
the responsibility of the client; the centralized font engine model has
failed. This is, says Jim, the biggest single architectural change
happening with X. Putting font support in the clients allows quicker
distribution of new technologies; Jim notes that there have been five
generations of font formats so far during the lifetime of X, and there is
no reason to expect things to slow down. Any change which requires X
server support cannot expect to see widespread deployment for at least two
years; client-side changes can propagate much more quickly.
One of the reasons for putting font support in the X server initially was
to keep bandwidth usage down. As it turns out, moving font support into
the clients has left bandwidth usage almost unchanged, but has eliminated
about 1/4 of the round trips to the server during application startup.
Anybody who has ever run an X application over a high-latency link knows
that those round trips hurt; eliminating them can only be a good thing.
Jim noted some lessons which come from the font experience. One is that
downward compatibility is absolutely necessary. New technologies (such as
newer font schemes) only
became accepted when libraries were provided to make them work with older
servers. No developer is interested in adopting technologies which
restrict the distribution of their applications. It is also necessary to
solve the full problem; in the case of fonts, that means solving the
problem for printing as well. With that in place, "even OpenOffice" is
moving over to the new font code.
Next point: eye candy is important. To a great extent, "eye candy" means
translucent windows in one form or another. The real use for translucence
is likely to be in window managers; once the new capabilities become
available, Jim expects yet another round of window manager
experimentation. "Another thousand flowers will bloom; many of them will
stink." But what survives will enrich the graphical experience for all of
Keith took the stage for one of his famous demonstrations. His desktop is
looking better all the time, with menus fading in and out and extensive
(some might say excessive) use of translucence everywhere. He pointed out
that translucence is a pixel-by-pixel property. He runs an xterm with a
translucent background, but the text is opaque, and casts solid shadows.
High bandwidth applications, such as full-motion video, run in a
translucent mode with no apparent performance problems - on a fast laptop,
"Eye candy" also means tools like screen magnifiers (which keep up in real
time) for people who have a hard time reading the screen's contents.
"Thumbnailers" (such as those often used for virtual desktops) will also
benefit from this work. Animation needs better support, and X needs better
integration into three-dimensional environments. "We live in a 3D world,
perhaps our work environments should be 3D as well." Eliminating
"flashing" (making desktop updates look nice) is also an important goal.
All of this is happening, in classic computer science manner, through the
addition of another level of indirection. X applications have typically
rendered their graphics directly into the frame buffer, unless they made
explicit use of off-screen pixmaps. In the new world, all rendering will
happen into off-screen memory. A new "compositing manager," which
typically works in conjunction with the window manager, renders the
off-screen windows onto the display, perhaps transforming them along the
way. It is this process which, for example, causes a newly maximized
window to fade, fully rendered, onto the screen, rather than just flashing
up blank and being redrawn in place. The policy for when and how such
effects should be used lives entirely within the compositing manager.
To accomplish all of this, four new extensions have been added to the X
protocol. "XFixes" is a sort of repository for minor repairs for things
that were done wrong in the original protocol. Among other things, XFixes
promotes Regions to a first-class X protocol object. The "Damage"
extension allows X
clients to be notified when the contents of windows changes; it can be used
by the compositing manager, screen magnifiers, thumbnailers, and any other
process which needs to know when things happen. Interestingly, the Damage
extension actually shrinks the size of the X server; a sign, according to
Jim, of something that should have been done that way from the beginning.
The "Composite" extension
handles the off-screen rendering of windows and the interactions with the
compositing manager. And "Xevie" allows the
interception of input events; it is used both for compositing (where event
locations may have to be transformed on their way to the client) and for
The latency issue is on the X developers' minds. X client startup can be
slow, especially on high-latency links. This slowness is caused by the
large number of round trips required between the client and the server
during initialization. Client-side fonts eliminated 25% of those, but, for
a typical client, some 130 round trips still need to happen before any
actual work can get done. Jim believes that 90% of those can be
eliminated; some will be harder than others. One of the biggest remaining
offenders, it turns out, is X extensions; apparently an "Extension
extension" may be required to batch together extension information.
In the 2D world, the Cairo
library, by Carl Worth and Keith Packard, is
starting to come into use. This engine, says Jim, brings "world-class,
two-dimensional, anti-aliased graphics" to the free software world; it is
second to none. Along with top-quality graphics, Cairo turns printing into
a first-class citizen; the audience reacted to this statement with loud
applause. There is work left to be done with Cairo, including the addition
of more backends and, inevitably, optimization.
Security is another issue. Currently, the only rational way to use X
remotely is through an ssh tunnel, but that has its costs: extra processes
in the loop, more latency, etc. X clients need to be able to talk directly
with the server in a secure manner. Jim notes that X terminals are coming
back. Maintenance of desktop clients is costly, and the deployment of
simple X terminals can reduce those costs. But it has to be possible to
use them in a secure manner.
The other interesting security issue is access to shared resources. It
would be nice to be able to walk up to a projector wall with a laptop and
start making use of it. Handling of input devices also figures into this
equation; how does one connect a mouse to a wall screen? If this problem
could be solved, there is the potential to "explode the PC into its
component pieces" and create an interesting, dynamic world.
There are no security policy options in X for this
kind of shared use, however. Jim briefly mentioned some ideas involving
integration with SELinux as a way of addressing this area.
A related topic is mobility. Wouldn't it be nice, asks Jim, if you could
leave the office, go home, and pick up with your applications on your home
system? This is a hard problem, but it is being addressed; there is
prototype migration code in GTK now. There are security issues to deal
with, of course, along with little things like teaching Xlib to deal
gracefully with connection loss.
Other issues mentioned in passing include proper network audio support
("all of the current audio servers are lame; we need one good one"), better
color management (for things like printers too), and proper support for
compression in the X protocol. It seems that the bulk of X traffic,
anymore, is image data, and much of that data compresses very well.
Finally, there are the packaging and development issues. Work to split the
X distribution into its component pieces (server, libraries, applications,
fonts) continues; there will eventually be no "tarball from hell" to deal
with. Modern building tools are being adopted; imake is on its way out.
Much of this work is done, but some pieces remain.
On the community side, Jim notes that X.org is a very different place from
what it was a year ago. Most of the XFree86 developers have come over, and
it is clearly the focal point for X development.
Perhaps the most important part of Jim's talk was repeated several times:
the X developers have ambitious aims and a great deal of work that needs to
be done; they are actively looking for developers to take on pieces of it.
For anybody out there who is looking for a project to support, X.org
represents a major opportunity. X is the core, the kernel, as it were, of
the Linux graphical environment. The development effort is healthy again,
and it has no end of interesting and challenging tasks to address. It is
hard to imagine a better, more interesting, more visible, or more important
project to work with. If you are looking to do some hacking, X would be an
ideal place to do it.
to post comments)