"Opt-in" metrics planned for Fedora Workstation 42

Red Hat, through members of the Fedora Workstation Working Group, has taken another swing at persuading the Fedora Project to allow metrics related to the real-world use of the Workstation edition to be collected. The first proposal, aimed for Fedora 40, was withdrawn to be reworked based on feedback. This time around, the proponents have shifted from asking for opt-out telemetry to opt-in metrics, with more detail about what would be collected and the policies that would govern data collection. The change seems to be on its way to approval by the Fedora Engineering Steering Council (FESCo) and is set to take effect for Fedora 42.

Details

The change proposal is owned by Allan Day and Michael Catanzaro. It says that the goal of metrics collection is to provide accurate data about the use of Fedora Workstation that will help accelerate development "in line with our users' needs and requirements". It is important to note that, as proposed, Workstation is the only Fedora edition or spin that would have metrics collection.

Initially Red Hat was not mentioned in the change proposal, but it was updated on the Fedora wiki after user "B Tb" asked for Red Hat's role in the proposal to be clarified. Christian Schaller elaborated on that on the Fedora Discourse discussion about the change, saying that he asked Day and Catanzaro to work on metrics after seeing Rob McQueen's talk about metrics in Endless OS at GUADEC in 2019:

What I want to see come out of this effort is enough data to both direct our development efforts towards what provides Fedora users the most value, help make technical decisions on things like which GNOME extensions to enable by default, help drive further investment from hardware partners and help drive more investment in Fedora in general. The overall goal for all of that is to see strong userbase growth for Fedora, both because as someone who has been using Fedora and [Red Hat Linux] exclusively as my desktop for 25 years I want to see it continue to prosper and because I believe that the more successful Fedora Workstation is the more successful Red Hat Workstation is.

Even though Red Hat has asked for the proposal, the project will be under the control of Fedora. The metrics collection will be run by a to-be-created metrics special interest group (SIG) as part of Fedora. The SIG would be responsible for packaging the metrics software and creating documentation for users. Naturally, the client and server software to be used for collecting metrics, to be based on the Endless OS Foundation's metrics system, is open-source. On his blog, Will Thompson explained how that system works in great detail.

According to the proposal, no personally identifying information (such as IP or email addresses) would be collected. The preliminary list of information that might be collected falls into several categories including hardware details, system settings, desktop-usage patterns, internationalization settings, and installed applications. This could include specific information such as the system's CPU, amount of memory, how many displays are attached, disk-partition configuration, display language, installed applications, and what media codecs are installed on the system.

In addition to information about the systems that Fedora Workstation runs on, the working group is after details about how the system is used. For example, how applications are launched, how often users switch between windows and the methods used to do so, the number of applications open, and much more. All of this is described as "generic, standardized information" that preserves user anonymity.

The proposal promises that identifying information, such as the web sites a user visits or which files are opened, will never be collected. Data would be filtered on the client before uploading to avoid collecting identifying information. For instance, the list of a system's installed packages would be filtered against a list of known packages. If a user has packages installed that do not match that list, then they would not be included in a metrics upload. To avoid user fingerprinting (the identification of a user based on the uniqueness of their set of metrics), each metric would be stored separately and would not be linked to other metrics from the same system. That is, information such as a system's CPU, display language, and memory usage would be tallied as discrete metrics rather than an entry describing a single system with those metrics.

Once metrics are in place, any changes to the system that affect the data collected, how the data is hosted, or the metrics SIG's governance, would have to be approved by FESCo. The proposal does not address how users would be notified when or if FESCo were to approve new metrics to be collected.

You must choose

The initial proposal's request to make metrics opt-out, rather than opt-in, was unpopular with many who took the time to comment. Fedora Project Leader Matthew Miller held a straw poll on Fedora's Discourse forum asking users which (if any) approaches they would accept to collect metrics. 530 users participated, 61% preferred an explicit opt-in, and only 15% chose an explicit opt-out (which was in the original proposal). Perhaps surprisingly, a mere 14% of the voters were entirely against any proposal to collect metrics.

Under the new proposal, metrics will be opt-in only, or at least users are forced to make a choice one way or the other. During initial setup, users will be asked if they consent to data collection. If users consent initially, they can disable metrics collection later in GNOME's system settings. Users will also be able to view the metrics that have been collected on their system, to understand what is being sent. Finally, users will be able to remove the packages responsible for metrics collection using DNF, though Catanzaro indicated that the package providing the metrics API to other applications would be a hard dependency for some applications. The package that submits metrics, however, will not be a hard dependency and users can easily uninstall that.

Of course, many users will be upgrading to Fedora 42 rather than doing a fresh install. For those systems, Neal Gompa said on the fedora-devel list that metrics will be off by default because there is no way to prompt the user to make the choice. Those who upgrade will be able to turn on metrics in the GNOME settings application, if they wish.

The raw data that is collected will not be shared with the rest of the Fedora Project or made public, by default. According to the proposal, this is because of the risk that non-anonymous data could be collected. "Out of an abundance of caution, we therefore only want to share data once it has been manually checked." Members of the metrics SIG will have full, ongoing access to the data. Members of the larger Fedora community will have to make a request for data to the SIG. If granted, the data will be shared privately. It is unclear what terms would be attached to receiving the data. The SIG is expected to publish data analysis, and the privacy and transparency checklist (which contains steps that must be completed before implementing the proposal) indicates that metrics will be reviewed and those that are not useful will no longer be collected.

Discussion

Zbigniew Jędrzejewski‑Szmek wrote that it was clear "a lot of effort was put into answering previous complaints in a very comprehensive way". User "boredsquirrel" wanted to know about the cross-desktop possibilities of the metrics system. Seth Maurice‑Brant responded that it should be possible to integrate metrics into other editions and spins, but that is currently out of scope of the proposal. Catanzaro said that it would be up to other editions to figure out in future change proposals if those developers wanted to collect metrics.

Vitaly Zaitsev objected to the collection of data that indicates what country a device is located in. He said that he was concerned that data might be "used against users from countries and regions that the US does not like (e.g. sanctions, export policies, etc.)". Catanzaro reminded that data points would not be aggregated with other data. He also said that users should not worry, yet, about specific metrics: "because each metric will need to be debated separately before it gets approved to be collected". He also said he did not see a strong reason to collect information about the country a user is located in.

Even though users have to opt into metrics collection, the proposal indicates that the yes/no toggle would have no default value: this means that users have to explicitly choose one or the other. Marc Deop i Argemí said that was unacceptable and the default position should be "off". "Most users will just click on 'Yes' without really comprehending what they are doing. And you _know_ this." Gary Buhrmaster replied that it was likely a matter of the phrasing of the prompt and wanted to know what the proposed wording would be.

Catanzaro said that they do not have the wording yet, but it would be something along the lines of "Help improve Fedora by sending anonymous usage data". The design of the question was critical, he said, "because if the acceptance rate is too low, then the project will fail and we're just going to be back here in a couple years" to debate making the metrics opt-out instead. Deop responded "you make it sound like you will never stop until you get the metrics you want".

FESCo votes

As of this writing, the proposal is not formally accepted but it is well on its way. According to FESCo ticket voting rules, if a change request receives at least three votes in favor and no votes against within one week, the change is approved without further discussion. If any FESCo member votes against the change, then it is added to the next FESCo meeting agenda for discussion and a majority vote would be required to accept the change. The change request has six votes in favor as of this writing, out of nine, and none against. Voting began on July 15, so the proposal will be approved on July 22 unless one of the remaining FESCo members votes against it. Even if a member objects and triggers a discussion, six votes for approval would easily clear the bar.

The next steps will be the formation of the metrics SIG and discussions about the final implementation of the system and metrics to be collected. Catanzaro has invited those who are interested in joining the SIG to contact him. "I hope to be wrong, but I expect it may be difficult to find people who are interested in participating." He later said that he had three volunteers, which was more than he expected, but still not enough.

There is still much work to be done, and discussion to be had, before any metrics will be collected. The current schedule for Fedora 42 has a deadline of January 14, 2025 for adding new features to Workstation, and February 25 as the deadline for removing any features that are not ready for the release. Final release is scheduled for mid-April 2025.

Note: you can avoid this step in the future by logging into your LWN account.