A discussion with Keith Packard

Since joining XFree86 a bit over three years ago, I've been working hard to rebuild X as a window system capable of supporting modern applications. Designed in 1987, X11 hasn't seen any significant architectural work in well over a decade, and it really shows.

Dirk Hohndel (while we were both at SuSE) encouraged me to build an extension that could expose the capabilities in modern graphics hardware to 2D applications, most notably anti-aliased graphics and image compositing. The Render extension is the result of that work.

This work was not done in a vacuum. Help from many people was critical in the design and implementation of all parts of the extension and supporting libraries. I've become deeply involved in KDE, Gnome, Mozilla and many other minor X projects. I've also been fortunate to have the assistance of people from the OpenGL community, the application community and one of the leading experts in computational geometry: Lyle Ramshaw.

Part of it is undeniably the steep learning curve for working within the X codebase; the project currently consists of more than 17 million lines of code, and a lot of the documentation necessary for writing device drivers is not available.

However, we cannot ignore the historical closed nature of XFree86 as another influence. Until I started at XFree86, the developer mailing lists and even the development version of the release in CVS was open only to people granted membership within the project, and such membership was granted only after demonstrating an ability to work with the code.

A certain amount of this was originally forced upon the project by the X.org contracts, but after the great licensing fiasco around X11R6, that reason disappeared and yet XFree86 remained a members-only project.

I pushed for changes in this policy, the result was a public CVS and some public mailing lists that I managed for quite some time. The old members-only developer mailing list remained, ostensibly for the purpose of discussions concerning non-public material, but in reality it was difficult to wean the existing members from a list not carrying traffic from new developers.

Another hurdle for new developers is the lack of any technical road-map for the project; releases are scheduled with no idea of what they will contain or even what people are working on. A new developer interested in helping out faces a difficult task in locating other developers even before attempting to learn the internals of the code. I attempted to suggest mechanisms to address this issue and was met with stiff opposition within the core team.

The XFree86 release schedule is entirely determined by the availability of the release manager. Because XFree86 is released monolithically, minor driver updates like support for new versions of existing chips wait for the next release rather than being made available separately.

I would like to see each library, driver and application released individually. X has a history of strong interface specifications and this should be leveraged in the release process. Much like Gnome or KDE is done today, occasional packaged releases could be done that look much like the current releases but which would be built by collecting releases of the individual packages together rather than attempting a unified release process.

Separating the release schedule of each module would make frequent updates possible where necessary while not burdening the downstream maintainers with huge volumes of complete X releases; areas of rapid progress would see rapid releases while stable code could be released more slowly.

Possibly even more importantly, splitting the release into modules would permit each module to be separately maintained and replaced; the project would be able to effectively leverage many more people in the management of changes. Many other projects do this today to allow scaling beyond the ability of a single individual to incorporate changes into the project.

There are several interfaces within the current X architecture:

• Protocol interfaces. These should be treated in much the same way as kernel/glibc interfaces where extrememly strong compatibility requirements ensure cross version interoperabilty.

• Library interfaces. These are the same as the application/library interfaces present today and require the same level of interoperability as glibc versions do.

• Driver interfaces. Because XFree86 encourages binary-only graphics card drivers, it should support that with strong binary compatibility guarantees. These are possible given the module loading mechanism in place, but are essentially untested by the development community as the whole system is built monolithically.

The only new interoperability requirement here is that the driver interface provide cross version compatibility, and this is already required to support binary graphics card drivers. Having it a regular part of the X environment will ensure that the interface is tested by all developers instead of only those few who produce the binary drivers.

I don't believe either KDE or Gnome have approached the XFree86 project as collective units, but individual developers from each project have interacted with XFree86 developers. One significant failing in X management today is the area of standardization. XFree86 plays little or no role in the maintenance of standards supported by the code, instead that role is left to the moribund X.org group. For KDE and Gnome, the result has been no progress in interoperability standards, most notably the inter-client communications conventions which form the basis for cooperation among applications, window managers and session managers.

KDE and Gnome ended up starting a new organization (freedesktop.org) to extend published X standards. Clearly, the responsibility for maintaining and extending that standard should lie within some X project, but the lack of a standards process within XFree86 has caused developers to take their ideas elsewhere.

X design must be done in a public venue with input from the projects layered on top. The current driver architecture within the server is a model of efficiency and performance when executing X protocol as presented by the x11perf benchmark. However, real applications take little advantage of that code as the X drawing model fits very poorly into modern graphical applications. X architecture should follow from application requirements, rather than benchmark racing.

One obvious first step is for X developers to start using modern applications; many key X developers wear the badge of 'twm' or 'fvwm' with pride. I switched to a mixture of KDE and Gnome desktops several years ago so that problems with those environments would be immediately evident as I changed code underneath them.

Documentation about active projects and developers would help new contributors locate appropriate groups; splitting into many separate projects would limit the size of the codebase in which individual contributions could be made and publication of release schedules along with intended content would all help make the development process less opaque. Perhaps the most important thing of all would be an active participation by the key developers in conferences, mailing lists and IRC channels to make their contributions more visible and understandable.

Over the last year or so, I have become convinced that XFree86 is headed for a success disaster -- widespread adoption of Linux on the desktop will cause a catastrophe for X development given the current lack of resources and inability to scale the project beyond its current bounds.

An urgent need to open up development and invite in new contributors while changing processes to make the existing contributors more effective seemed critical. I believed then that fixing the processes would avoid the impending disaster. However, my attempts to work within the existing structure have proven less than effective.

As I've attempted to open up the development process and prepare for a significant increase in Linux desktop deployments, I've run afoul of the current XFree86 leadership. My ability to contribute to the project was severely curtailed in stages, first administration of the public server, next permission to represent XFree86 to other groups, then administration of the public mailing lists, and finally CVS access.

I don't blame the current leadership for this problem; there are no guidelines or processes in place to deal with conflict in the development community. As the project has grown, it has failed to adapt to that growth by building formal structures necessary to govern a large and diverse group of people.

However, with my ability to effectively contribute severely restricted, I was forced to consider how to resolve the problems I faced. I discussed my dilemma both with members of the XFree86 core team and outside interests. I first learned that I was not alone in my views on XFree86 leadership; several prominent voices in the wider X community agreed with my assessment and encouraged me to find solutions.

The conclusions from this process were clear -- effective cooperation within the development community would require a government sanctioned by that community to run the X project as there was no other way to restore trust between the community and the leaders. The XFree86 by-laws provided no mechanism to force such a change on the project, and I was not sanguine that the XFree86 leaders would accept such change willingly, and hence I was forced to consider alternatives, even as I worked to find resolutions within the project.

Upon learning of my discussions, the XFree86 leaders were angry that I could privately disagree with their leadership while publicly supporting the project itself. The result was that I was summarily removed from the core team and subjected to censure on public mailing lists and web sites.

My goal is to promote development of the X window system. Right now, I don't believe it is possible for me to effectively do this within the structure of the XFree86 project. I have asked that they seriously consider changes that I feel necessary to restore public confidence in the project leadership and provide an environment where people can work together on the technical challenges facing the window system even if their individual goals are not identical.

The XFree86 leaders have only publicly discussed minor procedural changes which don't address the fundamental questions of authority, trust and governance. I hope they seriously consider these underlying issues instead of making only superficial changes in the project.

I held a conference call last Thursday with representatives from many projects and representatives from the XFree86 core team to discuss these issues. I'll be publishing detailed minutes of that call soon, pending agreement from the participants. The messages from that call have been taken to the XFree86 leaders and I expect to hear back from them shortly.

The highest priority should be to create an environment where people are encouraged to participate to whatever extent they are able, and where these contributors feel ownership of their code and pride in the community in which they work.

Actual projects should, as always, be directed by the contributors themselves. Personally, I will be working on ensuring that X applications have the support they need within the window system to provide the best user experience possible, and that the window system is a good citizen in the overall Linux environment.