Fedora 12 and unprivileged package installation

Fedora 12 was released on November 17 with the usual pile of new packages and features. By the sounds, it is a solid, well-received release. But one feature—unpublicized, undocumented, and turned on by default—has a number of Fedora users up in arms, leading to a huge thread on fedora-devel, in the bugzilla entry, and here at LWN. In short, the problem was that in the Fedora 12 default installation, regular users sitting at the console could install signed packages from any repository that the administrator has enabled.

Since the release, and all of the publicity and complaints, the maintainers of PackageKit have decided to remove the feature. Out of this controversy, though, are lessons for any project regarding security, transparency, and system defaults. There were no real complaints about the existence of the feature, rather it was the choice to make it the default, coupled with a lack of any notice of the change, that led to the outcry.

Unprivileged package installation

Non-root install is a convenience feature, and one that was supported somewhat differently in earlier Fedoras. From F9 through F11, the same effect could be achieved by using PackageKit to install a package, entering the root password, and checking a box to allow that user to install in the future without needing to enter the root password. The key difference is that in F12, no root password is ever required; the checkbox has been removed and is treated as if it was turned on for all users.

PackageKit goes to some lengths to determine that the user is logged on at the physical console of the machine before allowing non-root installs. It only allows installation of packages, not update or removal, and requires that the packages are signed by a key that has been installed by root. The only repositories that are allowed to be used are those that were previously configured and enabled by a root user.

The use case is for single-user (or all trusted user) systems, where the logged-in user is likely to be the same person as the root user. Some people evidently don't like having to enter the root password, or, worse, having to track down the person with the root password, when they install software. It is part of the effort to simplify the desktop experience, with package installation being considered a "routine" task that many users would like to be able to do without the extra password-entering step. But that has serious security implications.

So, why wasn't the previous behavior just propagated into F12? It turns out that the PolicyKit feature that was used (auth_admin_keep_always) was eliminated between PolicyKit 0.9 and 1.0, because it was considered to be a security problem. Because F12 uses PolicyKit 1.0, it makes it difficult to just revert to the old behavior. Instead, PackageKit maintainer Richard Hughes has decided to change the policy such that the root password will always be required for installing packages on F12. An alternative was proposed by Kevin Kofler that may allow the earlier behavior to return without the PolicyKit support, though it is unclear whether it is being considered.

Security ramifications

It didn't take much thought for various folks to come up with security issues with the new feature. Even for the stated use case, allowing unprivileged package installation has some fairly significant implications. The idea that running on the console somehow makes a process trusted is dubious at best. Firefox is an excellent example of a program that regularly has flaws which may lead to arbitrary code execution. That means that attackers on the web may be able to install packages on F12 systems.

But the proponents of this feature insist that there is no risk to installing trusted packages from trusted repositories. There are a number of problems with that thinking, starting with the fact that there are, without question, "trusted" packages in the Fedora repositories today that have privilege escalation and other security flaws. Trusting a repository does not in any way imply trusting every package in it. Administrators may well have added other repositories to pick up a package or two as well, without considering the fact that they have now opened up their systems to all of the packages contained in that, less trusted, repository.

There is also an incident that some are conveniently forgetting. In August 2008, there was some kind of break-in to the Fedora project servers, including the system used for signing packages. There is no evidence that malicious packages were signed at that time, but it is always a possibility in the future. During the time when Fedora was figuring out what happened, and issuing new signing keys, users were warned not to update their systems. But, if console applications can be subverted to do that installation, one can easily see a path to mass compromise of Fedora systems.

Because of the way this was changed, administrators who upgraded to F12 will find that the privileges of the users on their system have suddenly been elevated. Because PackageKit and PolicyKit are relatively new additions, many administrators may be largely unaware of them and their capabilities. Eliminating PackageKit is one way to avoid the current problem, but other, seemingly unrelated packages are dependent on it; setroubleshoot for example. Because PackageKit and PolicyKit function in ways that are very different from the traditional UNIX security model, it is imperative that Fedora protect its users from unexpected security holes, at least in the default installation.

SELinux hacker James Morris has a summary of the problems that can occur, including such things as denial of service from exhausting the disk space on the system. A user could also install an SELinux policy that is weaker than that installed by the administrator, leading to an overall reduction in the security of the system. Overall, the implications of this change were not fully explored before it was added to F12.

Defense

The main defense of the PackageKit behavior seems to be that any attacker who has physical access to the machine has many ways to subvert it. Hughes listed a number of other actions that a normal user can do in F12, some of which could certainly be considered security issues. Those don't make for a valid reason for the PackageKit changes, though, as Rick L. Vinyard Jr. points out: "Perhaps those should also be discussed and analyzed further, but that doesn't serve as a justification for the matter at hand."

There are multiple scenarios where console access does not imply access to the machine. A monitor, keyboard, and mouse are all that is required for a console, not necessarily access to the power button, USB ports, CD drives, and so forth. So, it is not universally true that console access equates to physical access. In addition, various tools like VNC allow remote users to act as if they were on the console. While VNC itself is detected as a non-local console, x11vnc is not.

Normally, one would expect this kind of change to be noticed in Rawhide—Fedora's development distribution—long before the F12 release. That would have allowed the issue to be discussed and resolved well in advance of unsuspecting users upgrading into the new policy. Unfortunately, the Rawhide packages aren't signed, so PackageKit always requires the root password to install them. So the first time the Fedora community saw this change was after they had upgraded to the "real" F12.

One might also expect a change of this magnitude to appear, perhaps quite prominently, in the F12 release notes, but that was not the case at the time of the release. Since that time, a very eye-catching note was added to the security section of the release notes. That could serve as a warning for users that read, or at least skim, those notes.

The other main line of defense is that this behavior is "just" a default, and can be changed by administrators. While that is true, the process to do so is not obvious. It involves mucking about with PolicyKit files, something that many Fedora users probably know little to nothing about. Hughes thinks that users should learn PolicyKit: "If you're deploying F12, then I really think you should know the basics about PolicyKit." But, when Seth Vidal set out to find out how to disable the feature—documented on his blog—asking Hughes did not lead to the solution: "So, if our engineers don't know the basics, how should our users?"

Overall, Hughes's reaction to the problem has been dismissive, bordering on rude:

Based on the above, one could argue with the "trivially", but, more to the point, one must understand an existing security model before changing it. If one myopically focuses on a single use case, and ignores the problems inherent in even that case, concluding that allowing unprivileged users to do package installation might make sense. But, for overall system security—not to mention the reputation of Fedora as a distribution that keeps security in mind—myopia is not a good strategy.

Hughes often refers to the change only being made for the "desktop spin", but that doesn't really make sense as the feature was added to all of Fedora. Certainly, some spins—server, for example—could change this default, but that seems backward. The core should default to secure choices, and allow spins to relax those requirements if they so desire. As Adam Williamson points out:

Larger issues

Regardless of how they feel about the specific feature in question, Fedora developers would like to avoid being blindsided by these kinds of changes in the future. To that end, Chris Adams started a discussion on security policy oversight: "Any package (whether new or an update) that adds/changes PolicyKit, consolehelper, or PAM configuration, and anything that installs new setuid/setgid executables, should require some additional third-party review." As part of that discussion, Hughes seems to be coming around to the majority view: "At the moment we're asking the server spin to essentially close the door, when maybe we should start with a closed door, and be asking the desktop spin to open it up a little more."

There is concern that a package maintainer can change global behavior for the distribution without any notice. Once the change is made, that maintainer can refuse to change it back, requiring the Fedora Engineering Steering Committee (FESCo) to step in and make a decision. In the case of a security problem, one that the maintainer is unwilling to acknowledge, the delay could be a serious problem. Since the change to PackageKit still has not been released as of this writing, there are numerous systems out there that are being installed or upgraded into less security.

FESCo discussed the issue at its November 20 meeting, and Vidal will be putting together a proposal to require the maintainers of critical packages to disclose any changes that might have this kind of impact. From the discussion of Adams's call for more oversight, a need for an overall framework of what users should and should not be able to do was noted. Fedora engineering manager Tom "spot" Callaway has started gathering a list of things that unprivileged users should not be able to do that would presumably factor into such a policy.

There were also some ancillary, but still important, issues. The responses from Hughes and David Zeuthen ("I'm not interested in this bike-shed or what color it is") in the thread angered quite a few. As the discussions grew, several Fedora leaders tried to tamp down the flames, which also didn't sit well with folks. A suggestion to vote on the bug, rather than continue piling on to the discussion was met with opposition as well, leading Jeff Garzik to note:

At one point Red Hat employees started using hidden comments in the bug to create a virtual "executive session", but folks started noticing the skipped message numbers and wondered what was going on. Williamson stepped in quickly to stop that:

The future

The Fedora project has likely learned quite a bit from this particular controversy, and it seems to be taking the right steps to avoid a repeat in the future. For a distribution that went through a great deal of pain to integrate SELinux features in order to increase the security of the system, it is mind-boggling to many that this non-root install feature was added as the default. There were multiple missteps—making it the default, not highlighting it in the release notes, not testing it in Rawhide, and so on—but those can all be corrected. Hopefully, the outcry and publicity will ensure that the word gets out, so that Fedora users will understand the issue and can make the appropriate changes for their systems.

In the meantime, though, other projects—distributions or software packages—would be well-served by studying this episode. Security is hard, and requires great diligence. It is likely that other projects could have hit this same kind of problem, but, hopefully, with this incident as a guide, will avoid doing so in the future.