Fedora evicts WolfSSL

The Fedora Engineering Steering Committee (FESCo) has voted to immediately remove the WolfSSL package from all of Fedora's repositories due to its maintainer failing to gain approval to package a new cryptography library for Fedora. WolfSSL's brief travels through Fedora's package system highlights gaps in documentation, as well as in the package‑review process. The good news is that this may stir Fedora to improve its documentation and revive a formal security team.

Fedora and cryptography

Fedora's packaging guidelines require that every application entering Fedora be checked for compliance with the policies on cryptography, but those policies could be written more clearly and are in need of an update. For example, the crypto policies say that new libraries "must comply with the crypto policies to enter Fedora" which seems oddly circular since the reader would likely think that is what they are reading. However, that is meant to be a reference to Fedora's crypto‑policies project, and that crypto libraries must have full integration with this system.

The crypto-policies project, maintained by Alexander Sosedkin, is an effort to unify the crypto policies for the whole distribution and also simplify the management of crypto applications and libraries. This means, in part, that Fedora has a limited set of approved crypto "back-ends" such as OpenSSL, GnuTLS, and Libreswan.

Fedora users can set system-wide crypto policies, such as a legacy policy when older encryption algorithms are needed for compatibility or the FIPS policy for conformance with FIPS 140 requirements. This system was adopted with the Fedora 21 release in 2014. The change proposal has a description of the system, and the crypto-policies man page describes the available policies and tools. A new crypto library would need to integrate with this system, but first it would have to be accepted in the first place.

Crypto libraries new to Fedora are required to get approval before being added, though the documentation does not do a great job of describing that process. Even though the packaging guidelines are a bit confusing, it should be clear enough that packagers need to consult with the Fedora security team, and then gain an exception from the Fedora packaging committee before being added to the repositories. There is one minor problem with this, though: the Fedora security team has been defunct for a while, and the policies have not been updated to reflect this.

WolfSSL

This posed difficulties for Andrew Bauer, who wanted to package WolfSSL, an SSL/TLS library written in C, as a dependency for another package he maintains, Netatalk. That software is an open-source implementation of the Apple Filing Protocol, which allows Linux (and other) systems to act as AppleShare file servers for new and old Apple systems. Really old, in fact—Netatalk version 2 supports systems as far back as the Apple II and version 3 supports Macs running Mac OS 7.5 (released in 1994) through today's Macs.

The Netatalk project had dropped OpenSSL in favor of WolfSSL with its 3.2.0 release in June because OpenSSL 3.0 had deprecated encryption algorithms that it needed to use for authentication with ancient Mac OS clients. This meant that Bauer could no longer rely on OpenSSL to fill the needs of the Netatalk package in Fedora, and needed to package WolfSSL. However, WolfSSL would need special approval to enter Fedora beyond simple package approval.

With Netatalk requiring WolfSSL, Bauer created a package and filed a review request on August 3. As part of the request, he included the rpmlint output that had flagged the package as non-compliant with Fedora's packaging policy. Bauer said:

Grepping the source code, one can see that wolfssl calls wolfSSL_CTX_set_cipher_list() rather than SSL_CTX_set_cipher_list(). Thus, this is a false positive and can be ignored.

And ignore it he did. So did Felix Wang, who reviewed the package and approved it on August 17. It might have helped if the error had been more explicit, or if it were listed among the common rpmlint issues in the package documentation. An issue was opened about the lack of description in 2023. The error is meant to alert the user that the application may not be using the system-wide crypto policy. FESCo member Fabio Valentini replied to the WolfSSL review‑request ticket to say that it would require further review and pointed to the crypto policies.

Security team snafu

So, Bauer sought help via the fedora‑devel list on August 18 to figure out how to contact the non-existent Fedora security team per the packaging guidelines. Neal Gompa responded that the "formal security team disbanded many years ago". He suggested that Bauer visit the Fedora security team room on Matrix, which Bauer did. (Note that link and several others require logging into a Matrix server. It is possible to log in without creating an account, click "Continue", select the Element client, and then click "Continue in your browser".)

He said that "a question was asked whether this package requires approval from Fedora Security". He got a quick suggestion from Demi Marie Obenour about compilation options that did not address the question of approval directly. He thanked Obenour, and seems to have left the channel before he received messages about Fedora's crypto policies or suggesting that new crypto libraries would require a change request. Bauer plunged ahead and updated the review ticket to report he'd been given advice on building WolfSSL, and was requesting a package repository now. The WolfSSL package entered the testing repositories for EPEL 9, Fedora 39, 40, 41 (which is still in development), and rawhide on August 20.

Valentini replied to Bauer's email on fedora-devel saying that the question of whether WolfSSL complies with system crypto libraries had not been answered. He was unhappy that it had already been imported into Fedora. Daniel P. Berrangé wrote that it seemed non-compliant and that a Fedora packaging committee (FPC) exception is required.

More than two weeks later, Valentini followed up in the review ticket and said it looked like Bauer had ignored his earlier comment about crypto compliance completely. The two went back and forth a bit. Bauer again complained the documentation was unclear and said: "Once we identify the source of authority, looks like that's Felix, I'll be glad to [work] through this."

Retirement

Instead of debating further with Bauer, Valentini opened a ticket with FESCo about the matter. He noted that "the reviewer of the package (not the submitter)" had filed a request with the packaging committee to approve the package, but it had already made its way into Fedora's stable repositories.

Stephen Gallagher said that he'd spoken with the crypto-policies maintainer and members of Red Hat's security team. Their opinion was that WolfSSL should not be permitted in Fedora without honoring crypto-policies. Gallagher proposed that FESCo vote on the following:

WolfSSL is immediately retired from Fedora. The maintainers may file a new package review request when WolfSSL respects the crypto system policy. This review request must be presented to the FPC, who must approve it before it is added back to the repositories.

On September 10, Valentini wrote that Gallagher's proposal had been discussed, voted on, and approved by FESCo. Five votes were in favor of the proposal, none against or abstaining.

Typically, packages are only retired from rawhide or branched versions of rawhide that are in the process of becoming the next stable Fedora release but have not entered final freeze. Retiring packages from stable versions of Fedora is a rare step, and usually only undertaken when a package has licensing problems. Once the package is retired, it is removed from subsequent composes and will be removed from the archives. It will not be removed from systems it was installed on unless it is also added to the fedora-obsolete-packages package. That has not been done in this case. As of this writing, the package is still installable, but it should be disappearing soon.

Aftermath

There are several points when Bauer clearly should have slowed down and gotten the message that he needed to consult Fedora's packaging committee. If nothing else, Valentini was waving a caution flag that Bauer should have heeded.

Still, he is not wrong that the documentation leaves much to be desired in terms of clarity and completeness. And, as Matthew Miller noted in the FESCo ticket, the security team is in a "disordered state" and it may be time to "reorganize and re-formalize the security team". Michel Lind commented that he had been sounding out frequent posters in the Matrix security room about improving things, so the project may be gathering steam to put a real team in place, once again, to avoid a recurrence of non-packagers giving random advice that is taken as official. It might also behoove Fedora to consider whether a chat room is a real substitute for an email list that does not emphasize real-time communication or require a special client to use.

Fedora's policies may also need some updating to take into account use cases that require special consideration for compatibility with obsolete hardware and software. It is a net good that Fedora users can use software like Netatalk to connect with classic computers, and it would be a shame if Fedora's policies blocked such software permanently. In the end, no real harm has been done, and Bauer's over-eagerness has helped the Fedora project identify a few real problems to address.