Freedesktop.org: its past and its future

At the 2018 X.Org Developers Conference (XDC) in A Coruña, Spain, Daniel Stone gave an update on the status of freedesktop.org, which serves multiple projects as a hosting site for code, mailing lists, specifications, and more. As its name would imply, it started out with a focus on free desktops and cross-desktop interoperability, but it lost that focus—along with its focus in general—along the way. He recapped the journey of fd.o (as it is often known) and unveiled some idea of where it may be headed in the future.

The talk was billed with Keith Packard as co-presenter, but Packard could not make it to XDC; Stone said that he sent Packard a copy of the slides and heard no complaints, so he left Packard on the slide deck [PDF]. Stone wanted to start with the history of fd.o, because there are lots of new contributors these days—"which is great"—who may not know about it.

Yesteryear

Fd.o was founded in 2000 by Havoc Pennington, who is a GNOME developer; he was joined by a few more developers from other desktops. The initial goal was to provide a forum for discussing common desktop standards. If you were running KDE applications under GNOME and using Sawfish as a window manager, "you wanted it to be coherent". Early on, the group came up with agreements on various behaviors, including drag-and-drop and copy-and-paste.

It was a "good time for desktop collaboration", Stone said. Further discussions led to agreements of MIME type handling, which gave us ways to specify what applications would be opened for different file types. In addition, theming agreements allowed customizing the look of users' desktops in a cross-desktop-friendly way. Eventually, fd.o grew support for hosting source trees managed by the CVS revision control system, so it was not simply a "place to dump some HTML specifications".

In parallel to this, a short-lived site called xwin.org was started in 2003. At that time, the X Consortium "essentially did not exist"; all X development was done internally by companies like Sun and HP or by the XFree86 project. When Stone tried to package XFree86 for Debian, he could not get access to the source repository; the development mailing list was closed to outsiders as well.

Eventually, XFree86 changed its license to be hostile to other projects. At that point, Packard and some others started xwin.org to discuss what to do. "Everyone jumped on board" the xwin.org train at that point. From his perspective as a packager, it was great to be able to talk directly with the developers, he said.

At that point, X was governed by The Open Group (TOG), which offered to host the discussions about the license change and any subsequent plans. The name of that organization is rather ironic, Stone said, since it is a closed, "pay for play" group. In any case, the xwin.org developers merged the TOG X11R6 tree and the last release of XFree86 with a reasonable license; that became X11R6.7. Along the way, the X.Org Foundation was formed and got its independence from The Open Group; he doesn't know if the TOG was just being nice or whether it was "happy to see the back of us", he said with a grin.

At that point, it made sense for what was xwin.org to merge with fd.o due to the "shared interests" of the two; fd.o absorbed all of the xwin.org projects and communities. The X code had forked away from XFree86 and fd.o was hosting all of the X.Org projects, which is still the case today. Meanwhile, the fd.o standardization efforts continued with the "cross-desktop group" (XDG) family of specifications. Out of those efforts came things like .desktop and Autostart files. It was a steady evolution of the kinds of things fd.o had been working on since it was formed.

"It started getting really out of hand" in 2004. CVS hosting was a painful thing for projects to handle on their own, so they were looking for places to put their code; fd.o would accept "basically anyone" onto its servers. It became something like SourceForge. At the same time, though, there was an idea to turn all of the specifications and projects that were being hosted into an "LSB-like platform for desktops". That effort failed; it was 12 or 13 years too early, Stone said, as Flatpak is mostly what was envisioned back then.

In 2006, fd.o was "a bit bereft of direction". It was also a busy time in X development, so Stone and Packard were busy hacking on X and did not have time to look at much else. There was little coherence between the projects that fd.o was hosting. In addition, fd.o had taken on communities that had little or no contact with fd.o as an organization. By 2009, fd.o got "very lost". Anyone who had vaguely expressed interest in helping administer the site was given root privileges. There was no way to discuss project-specific problems; in some cases there was no person to even contact about a particular project. The fd.o project was being run as an anarchistic cooperative; "member" projects were off doing their own thing.

The project started to degenerate; its infrastructure became quite unreliable. There were disk and machine failures that goaded him and others into action. In 2012, Tollef Fog Heen was sponsored part-time for system administration work. Stone started to work on some system administration and Packard began working with the community of projects more actively. Others also helped to administer the fd.o infrastructure.

A few years down the road, the project was a bit "dazed and confused". GitHub was well established by 2015 and it was unclear what fd.o offered over GitHub, GitLab, and other similar hosting sites beyond that fd.o was "something vaguely about the desktop". Projects hosted at fd.o were left alone with little or no communication from the fd.o umbrella project. All of the time that fd.o had was spent "treading water and paying down technical debt". Various projects had moved away from fd.o years before, but had never told fd.o about it; it was "fair enough, we had not talked to them in that time" either, Stone said. But it was also a time where fd.o started experimenting with new services such as Jenkins and Phabricator.

2017 was an inflection point for fd.o, he said. The infrastructure was stable and the administrators were "aggressively making things work". There was finally time and bandwidth to start thinking more about the fd.o community. There were worries about legal liability, so work started on adopting a code of conduct. Clear copyright violations, "things that would obviously get my door kicked in", were removed from the site. At the time, GNOME was looking around for hosting software for its code; some long discussions with GNOME developers about GitLab took place. Eventually, GNOME decided to use GitLab and fd.o followed suit.

Today

So, now in 2018, fd.o is an open, neutral collaboration space; there is no danger that fd.o will ever be acquired, Stone said. It has a loose coalition of projects and is made up of free services of various sorts. Fd.o has "consistently had good intentions" over the years, but it has a mixed track record in terms of execution.

Depending on how you count, fd.o has around 42 active projects at this point; all are "fairly active and healthy". There are 28 dormant projects, which means there has been five years or more with no activity in their Git repositories. There are also 30 extinct projects that never moved from CVS to Git and 31 departed projects, 24 of the latter moved to GitHub.

These days, fd.o can offer "all the usual services you expect" for project hosting. That includes code hosting, issue tracking, web pages, and mailing-list hosting. It does not, yet, have continuous integration (CI) features or a modern code-review tool. In addition, fd.o offers apologetic responses to requests it can't fulfill at this point; the fd.o administrators will be "quite nice" when they have to say no, Stone said.

Some modernization is in order, however. No one really seems to like Bugzilla and some projects do not want to do patch review on a mailing list. Beyond that, CI is not really optional anymore. Bugzilla and the lack of CI were the main reasons that projects moved from fd.o to GitHub, he said. In addition, adding an account to fd.o services required filing a bug and waiting for an administrator to make it happen; that will be changing as well.

As part of the long discussion with GNOME, Phabricator was considered, but it had some user-interface issues and looked like a project that could become more closed over time. GitLab is an open project that is run using an open bug tracker and code repositories. The GitLab developers are willing to hear suggestions from GNOME and fd.o and genuinely care about open source and open projects, he said. It also "runs really well, which helps".

The existing fd.o machines are not up to running GitLab, however. The company behind GitLab, GitLab Inc., has "quite generously offered to sponsor" fd.o for a "year-ish" to get it up and running in the Google cloud. In that time frame, fd.o will be looking for other sponsors to continue running there. "Now I know what the cloud is", Stone said with a chuckle. Kubernetes is "very complex" but he got things working by trial and error. Over the years, Portland State University (PSU) has been quite generous in hosting fd.o, but it is time to move on.

At this point, almost all of the projects have been migrated to GitLab. There are still some areas that need work, including the GStreamer specifications that need to be moved over and a couple of small projects that will be migrated soon. Beyond that, the Direct Rendering Manager (DRM) and kernel mode setting (KMS) projects must wait until the end of the year. The status of the migration is being tracked in an fd.o GitLab issue.

The disk space for fd.o is around 35GB of Git repositories, plus 7GB of Git large-file storage (Git-LFS); CI artifacts weigh in at around 7GB as well. There is 800MB of file uploads. The Docker registry image for CI is 157GB, however. There is less network traffic than he thought there would be, but once the kernel pieces migrate, that will change. For now, the EMEA region uses the most bandwidth at 200GB/month; the US is second at 180GB. That's another reason that moving the servers away from the west coast of the US (PSU) makes sense. He was happy to see that his native country, Australia, was represented in the traffic, using about 5GB/month.

Running the GitLab servers costs around $500/month, which is being covered by GitLab Inc. for now. And fd.o has a few thousand dollars in the bank. Those figures are about as organized as fd.o gets about financial transparency to the community, which is "not great" and needs to be fixed, he said.

Tomorrow

The project needs to clearly define what it is and does—and why. The obligations of fd.o to its member projects, and vice versa, need to be clearly laid out. Perhaps it should not just be open to any project that has stuck it out over the years. There should be an electable board, for example, which is something that the X.Org Foundation has had for a long time. X.Org has existed since 2004 and it is also a member of Software in the Public Interest (SPI). X.Org does great work. It also has great conferences, Stone said; the audience enthusiastically followed the suggestion of "applause" for that in his slides.

So he put up a strawman proposal for fd.o to join forces with X.Org. That leads to some open questions, though. X.Org has a narrowly defined mission (it is good at saying what it does, he said) that does not include all of what fd.o does. There are fd.o projects like ModemManager, Poppler, and (formerly) LibreOffice that are pretty far outside the scope of what X.Org does. Should the X.Org mission be widened or should some fd.o projects be excluded? As long as it is clear what projects can expect from fd.o (and, perhaps, X.Org), there are alternatives for most of the services that are provided.

There is also a question of how much independence fd.o would maintain. If a project is interested in being hosted there, who decides whether to allow it (or not)? Otherwise, moving fd.o under X.Org seems to make quite a bit of sense, he said. It would solve all of the community issues that fd.o has had without duplicating all of what X.Org has done.

No matter how that plays out, there is a division of responsibilities needed. A board cannot do it all, but volunteers don't always work out either. Two specific areas need to have people with delegated power and responsibility: infrastructure administration and handling code-of-conduct reports. Currently, the fd.o code-of-conduct committee consists of Stone, Packard, and Fog Heen, but fd.o wants projects to be enforcing their own codes; the committee is simply meant to be the final place for escalating problems that cannot be resolved by the projects.

There is also the question about transparency in handling code-of-conduct reports. The recent uproar surrounding the kernel code of conduct highlights the need to determine what should be published, how often, and where. As a starting point, he gave attendees a transparency report for 2018. The fd.o committee has handled three "non-troll" abuse reports so far this year. No action was taken on two of those, but there was a private discussion with the person reported in the other. In that discussion, it was suggested that the person might want to change how they deal with other people. No further action has been taken in that case.

As with most projects, fd.o can use some help. With a grin, Stone asked: "Have you seen our web site?" The cloud is still something of a mystery and more help would be useful there. The specifications and their current status is not really reflective of reality at this point, which could use a clean up. Helping the member projects get up to speed on the best practices would be helpful. There are fd.o issues in GitLab that need addressing as well.

The Q&A session was replete with various "thank-yous" for all of what fd.o has done over the years. When asked what he thought fd.o would be doing beyond just GitLab hosting in 3-5 years, Stone pointed to mailing-list hosting. Having a high-volume mailing-list server in 2018 is bad: "the internet hates you". That is something that fd.o will probably still have to handle itself.

The X.Org Foundation is looking at making some fairly small by-law changes to allow it to take on fd.o, foundation secretary Daniel Vetter reported later in the conference. The wording of that is being worked on; once that is complete, it will go out to members for a vote. That vote might take place before the end of the year, Stone told me after his talk, so we may know soon if "freedeXtop.Org", as he jokingly called the combination in his slides, is going to come about or whether fd.o will need to find another path to better organization, more transparency, and a thriving community. Whatever the result of the merge proposal, it is good to see freedesktop.org emerging from years in the wilderness.

[I would like to thank the X.Org Foundation and LWN's travel sponsor, the Linux Foundation, for travel assistance to A Coruña for XDC.]