Two visions for the future of sourceware.org

The session in question was initially set up as a birds-of-feather gathering where Sourceware "overseer" Mark Wielaard would describe some of the new services that are being developed for that site. He did not get far before being interrupted by David Edelsohn, who questioned whether it was correct to describe Sourceware as a "software project". Wielaard tried to push on, noting that there are currently two Red Hat employees, helped by a number of volunteers, looking after the site. Carlos O'Donell repeatedly broke in to describe Sourceware as specifically a Red Hat system. The site's mission statement, he said, describes it as "a Red Hat system providing services for open-source projects for Red Hat" (which isn't quite the wording on the actual statement). The purpose of these interjections was evidently (at a minimum) to dispute the notion that Sourceware is a community resource.

Continuing, Wielaard claimed that all seems well with Sourceware; the hosted projects are doing well and new services being set up. Sourceware does not really have any problems currently, he said. Wielaard was intending to talk about those new services (the slides for the intended talk are online), but he was interrupted at that point and reminded of an agreement, evidently made ahead of the session, that the real focus of the session would be the SFC proposal and it's (still undisclosed) competitor. Wielaard acquiesced after complaining that he really wanted to talk about new services. [Update: it's worth noting that the existence of this agreement is disputed by some of those who were said to be a party to it.]

The SFC proposal

With regard to the SFC proposal, he said, the idea had been posted on the mailing list and discussed there; the replies seemed to all be positive. O'Donell jumped in to say that the SFC proposal talked about fiscal management, but said nothing about who would be raising funds. That is a difficult and thankless task, he said. SFC director Karen Sandler, dialed in remotely, said that SFC would help with that task; that sort of support is something that SFC routinely provides to its member projects. That said, the SFC does rely on the projects themselves doing fundraising work as well.

Wielaard said that, in the discussion, representatives of the projects hosted on Sourceware seemed to like the SFC proposal, and the Free Software Foundation also seems to be happy with the idea. He said that he did not get a single negative response during the discussion and that his impression was that the community had accepted the idea; yelling from the audience made it clear that there was some disagreement at this point — disagreement that had never been expressed during the month-long mailing-list discussion.

O'Donell repeated that the SFC provides a place to put funding, but that it is necessary to find sources for those funds. Once that happens, he asked, what was the plan for using those funds? Wielaard answered that it would be good to have an offsite backup for Sourceware. Red Hat could turn evil someday (though he sees no signs of that now) and it is good to have a contingency plan. But there are no immediate needs for funds to keep Sourceware going; the purpose is to have a structure to receive funds should the need arise.

GTI revealed

At this point O'Donell, who along with Edelsohn clearly had another plan in mind, finally revealed it to the rest of the group. While discussion of new services for Sourceware would be great, he said, Cauldron is the place where the community can work on structural issues, such as where funding comes from and what is done with it. The SFC would be "within its rights" to take on Sourceware, it has many well-served projects and there is nothing wrong with it. But, he said, there are some serious challenges coming for the toolchain community that may require support beyond what SFC can provide. Specifically, he talked about cybersecurity requirements that could lead to the exclusion of the GNU tools from important projects if its development systems are not seen as being managed in a sufficiently secure manner. Details on what those requirements would be, or how they might be met, were not provided.

The Free Software Foundation is the fiscal sponsor for the GNU project, he continued, while Red Hat is currently the sponsor for Sourceware. Perhaps it is time to seek new sponsors; specifically, the Linux Foundation's Open Source Software Security Foundation (OpenSSF) could become the sponsor for a "GNU Toolchain Initiative" (GTI). There are useful things that could be done with funding, such as setting up Big Blue Button (BBB) servers or establishing a fund to support developers' travel to conferences like Cauldron.

But he would particularly like to see the toolchain's repositories and related services move onto "managed infrastructure", improving security while freeing up developer time for other tasks; this move is a big part of the GTI proposal and where a lot of the funding would go. The toolchain development model looks a lot like the Linux kernel's model, he said, so there would be value in taking advantage of the infrastructure that the Linux Foundation has set up to support the kernel. This is not about the governance of the toolchain projects, which would not change. Those projects could keep their development model and tools while offloading the administrative work.

The SFC's Bradley Kuhn, also dialed in remotely, said that he welcomes debate over how Sourceware should be managed. But naturally he felt that the SFC would be a better home; it is organized as a charity, he said, which gives it more of a community focus. He would understand if the toolchain developers were reluctant to hand off Sourceware to "a big IT team" controlled by large companies. Edelsohn responded that this proposal was not about control, it was about finding the best combination of funders and organizations to ensure the future of the GNU toolchain projects. The GTI proposal is put forward by a number of vendors working on those projects, along with maintainers, release managers, and more.

At this point there was a break in the discussion as it moved to a smaller room.

Governance

Jeremy Bennett restarted things by letting it be known that his company (Embecosm) is one of those that have agreed to sponsor GTI; there are evidently several other sponsors lined up, but none of them were named in the session despite requests. SFC is a fine organization, Bennett said, but he felt that the Linux Foundation would be a better home for Sourceware. He mentioned complaints about the proprietary conferencing system used for Cauldron, claiming that the community lacked the resources and skills to set up a BBB server to use instead. Having a funded operation to manage the infrastructure currently provided by Sourceware could be a way to address problems like that.

O'Donell said that this was the right time to have this discussion, not during the preceding period (seemingly over a year) that he and Edelsohn have been quietly working on GTI. He has been "burned" in the past by overly public discussions, so the development of GTI was, instead, done by creating a "stakeholder map" and working through it slowly and privately. Issues like where the money would come from needed to be worked out before going public, he said.

I could not resist taking this moment to make a bit of a speech. From my point of view, having the Linux Foundation take over kernel.org has been an unambiguously good thing for the kernel community; everything works well and the people running it are trustworthy. But, I said, the secretive way that this project has been handled is a poor example of how to deal with a community. Regardless of what the reality may be, GTI looks to a number of the people involved like a murky, hostile takeover of an important piece of community infrastructure. I asked what the governance structure for GTI would be, and finished by saying that, while the Linux Foundation does many things, supporting free software for remote conferencing isn't one of them; anybody hoping for support for BBB servers from that direction may be disappointed.

Sandler said that the SFC would happily step back if that's what the community wants. Frank Eigler, another one of the Sourceware overseers, said that, however things, go, future discussions on this topic need to happen in the open.

O'Donell started addressing the governance issue by saying that GTI would have an "open and transparent" technical advisory committee made up of the maintainers of the projects hosted there. Wielaard jumped in to complain that the openness is limited when proprietary tools are used in the current GTI discussions. O'Donell said that the use of these tools (seemingly Google Groups) was the result of "laziness" on his part and thanked Wielaard for pointing that out. Serhei Makarov observed (over the videoconference chat) that "laziness is not a good thing to demonstrate when proposing a major community move".

Edelsohn acknowledged that the GTI effort had been going on for a long time outside of the public eye but, he said, the point was not to present the result as a fait accompli. This organizing was happening during the pandemic and, more importantly, during a period where there were a number of controversies surrounding the Free Software Foundation. There was a strong desire for GTI to not look like an attack on the FSF or as a response to the debate surrounding it. Instead, the intent was to get a minimum number of sponsors to launch the project before going public, and to "be kind" to those sponsors who wanted their names to be on the initial press release. There is, he said, $400,000 in annual funding that has been committed at this point.

He also briefly mentioned the GTI governing board, which would be making the decisions on how to use those committed funds; I asked him to elaborate on that aspect of the plan. This board, he said will be populated by representatives of the "major sponsors" of the GTI, and will give one seat to a representative from the technical advisory committee. He added that "everybody" would be able to attend board meetings as observers; it wasn't clear whether he meant "everybody" or "everybody on the technical advisory committee".

Kuhn said that this plan constituted governance by the project's corporate members; that represents a major difference with regard to how SFC manages its projects. There, he said, projects are governed by a volunteer committee and funding organizations have no say in decisions. Bennett said that Sourceware is currently owned by Red Hat, so it is Red Hat's asset (and thus presumably governed by a company as well); Wielaard answered that, for a number of projects, Red Had just provides servers but is not involved in how they are managed. There was some disagreement in the room on this point, since Red Hat also funds some of the developer time that goes into running Sourceware.

Security and conclusion

At times during the session there were questions about which security issues the GTI organizers were worried about, and whether Sourceware has security-related problems now. Joseph Myers reminded the group of the 2010 Sourceware compromise, where a spammer was able to exploit outdated software running on the system to gain shell access and put up some web pages that had little to do with free software. The problem was discovered when the changes tripped up a cron job. Investigation revealed that the Apache server was running in the "gcc" group, meaning that the attacker had been in a position to modify the GCC Subversion repository. In that case it was possible to verify the integrity of the repository but, he said, we cannot assume that something worse won't happen in the future. Notably, nobody mentioned anything about the existence of any more recent security incidents.

Wielaard said that the SFC, too, had suggested putting some resources into security; O'Donell said "let's raise funds for it".

At this point the discussion had gone on for two hours and there was little desire to continue. Jose Marchesi asked what the next steps should be. Wielaard suggested that everything should be put onto the overseers list for public discussion. It should be possible to work things out as a community, he said, but he said he was no longer sure that the toolchain developers were still all one community. Elena Zannoni suggested that a technical discussion on security issues could certainly be held in public.

As of this writing, there still has been no public posting of the GTI proposal.

This article has tried to distill the important issues out of a discussion that was loud, contentious, and bordered on physical violence at one point. Most of that is not truly relevant; the important issue is that the GNU toolchain community has to find a way to rebuild trust and engage the full community in designing a future for Sourceware. This community, one of the oldest in the free-software world, has gotten through many difficult periods in the past; it should be able to handle this one as well.

The video for the first half of this discussion is on YouTube; the second half has not been posted as of this writing.

[Thanks to LWN subscribers for supporting my travel to this event.]