|
|
Subscribe / Log in / New account

"Opt-in" metrics planned for Fedora Workstation 42

By Joe Brockmeier
July 22, 2024

Red Hat, through members of the Fedora Workstation Working Group, has taken another swing at persuading the Fedora Project to allow metrics related to the real-world use of the Workstation edition to be collected. The first proposal, aimed for Fedora 40, was withdrawn to be reworked based on feedback. This time around, the proponents have shifted from asking for opt-out telemetry to opt-in metrics, with more detail about what would be collected and the policies that would govern data collection. The change seems to be on its way to approval by the Fedora Engineering Steering Council (FESCo) and is set to take effect for Fedora 42.

Details

The change proposal is owned by Allan Day and Michael Catanzaro. It says that the goal of metrics collection is to provide accurate data about the use of Fedora Workstation that will help accelerate development "in line with our users' needs and requirements". It is important to note that, as proposed, Workstation is the only Fedora edition or spin that would have metrics collection.

Initially Red Hat was not mentioned in the change proposal, but it was updated on the Fedora wiki after user "B Tb" asked for Red Hat's role in the proposal to be clarified. Christian Schaller elaborated on that on the Fedora Discourse discussion about the change, saying that he asked Day and Catanzaro to work on metrics after seeing Rob McQueen's talk about metrics in Endless OS at GUADEC in 2019:

What I want to see come out of this effort is enough data to both direct our development efforts towards what provides Fedora users the most value, help make technical decisions on things like which GNOME extensions to enable by default, help drive further investment from hardware partners and help drive more investment in Fedora in general. The overall goal for all of that is to see strong userbase growth for Fedora, both because as someone who has been using Fedora and [Red Hat Linux] exclusively as my desktop for 25 years I want to see it continue to prosper and because I believe that the more successful Fedora Workstation is the more successful Red Hat Workstation is.

Even though Red Hat has asked for the proposal, the project will be under the control of Fedora. The metrics collection will be run by a to-be-created metrics special interest group (SIG) as part of Fedora. The SIG would be responsible for packaging the metrics software and creating documentation for users. Naturally, the client and server software to be used for collecting metrics, to be based on the Endless OS Foundation's metrics system, is open-source. On his blog, Will Thompson explained how that system works in great detail.

According to the proposal, no personally identifying information (such as IP or email addresses) would be collected. The preliminary list of information that might be collected falls into several categories including hardware details, system settings, desktop-usage patterns, internationalization settings, and installed applications. This could include specific information such as the system's CPU, amount of memory, how many displays are attached, disk-partition configuration, display language, installed applications, and what media codecs are installed on the system.

In addition to information about the systems that Fedora Workstation runs on, the working group is after details about how the system is used. For example, how applications are launched, how often users switch between windows and the methods used to do so, the number of applications open, and much more. All of this is described as "generic, standardized information" that preserves user anonymity.

The proposal promises that identifying information, such as the web sites a user visits or which files are opened, will never be collected. Data would be filtered on the client before uploading to avoid collecting identifying information. For instance, the list of a system's installed packages would be filtered against a list of known packages. If a user has packages installed that do not match that list, then they would not be included in a metrics upload. To avoid user fingerprinting (the identification of a user based on the uniqueness of their set of metrics), each metric would be stored separately and would not be linked to other metrics from the same system. That is, information such as a system's CPU, display language, and memory usage would be tallied as discrete metrics rather than an entry describing a single system with those metrics.

Once metrics are in place, any changes to the system that affect the data collected, how the data is hosted, or the metrics SIG's governance, would have to be approved by FESCo. The proposal does not address how users would be notified when or if FESCo were to approve new metrics to be collected.

You must choose

The initial proposal's request to make metrics opt-out, rather than opt-in, was unpopular with many who took the time to comment. Fedora Project Leader Matthew Miller held a straw poll on Fedora's Discourse forum asking users which (if any) approaches they would accept to collect metrics. 530 users participated, 61% preferred an explicit opt-in, and only 15% chose an explicit opt-out (which was in the original proposal). Perhaps surprisingly, a mere 14% of the voters were entirely against any proposal to collect metrics.

Under the new proposal, metrics will be opt-in only, or at least users are forced to make a choice one way or the other. During initial setup, users will be asked if they consent to data collection. If users consent initially, they can disable metrics collection later in GNOME's system settings. Users will also be able to view the metrics that have been collected on their system, to understand what is being sent. Finally, users will be able to remove the packages responsible for metrics collection using DNF, though Catanzaro indicated that the package providing the metrics API to other applications would be a hard dependency for some applications. The package that submits metrics, however, will not be a hard dependency and users can easily uninstall that.

Of course, many users will be upgrading to Fedora 42 rather than doing a fresh install. For those systems, Neal Gompa said on the fedora-devel list that metrics will be off by default because there is no way to prompt the user to make the choice. Those who upgrade will be able to turn on metrics in the GNOME settings application, if they wish.

The raw data that is collected will not be shared with the rest of the Fedora Project or made public, by default. According to the proposal, this is because of the risk that non-anonymous data could be collected. "Out of an abundance of caution, we therefore only want to share data once it has been manually checked." Members of the metrics SIG will have full, ongoing access to the data. Members of the larger Fedora community will have to make a request for data to the SIG. If granted, the data will be shared privately. It is unclear what terms would be attached to receiving the data. The SIG is expected to publish data analysis, and the privacy and transparency checklist (which contains steps that must be completed before implementing the proposal) indicates that metrics will be reviewed and those that are not useful will no longer be collected.

Discussion

Zbigniew Jędrzejewski‑Szmek wrote that it was clear "a lot of effort was put into answering previous complaints in a very comprehensive way". User "boredsquirrel" wanted to know about the cross-desktop possibilities of the metrics system. Seth Maurice‑Brant responded that it should be possible to integrate metrics into other editions and spins, but that is currently out of scope of the proposal. Catanzaro said that it would be up to other editions to figure out in future change proposals if those developers wanted to collect metrics.

Vitaly Zaitsev objected to the collection of data that indicates what country a device is located in. He said that he was concerned that data might be "used against users from countries and regions that the US does not like (e.g. sanctions, export policies, etc.)". Catanzaro reminded that data points would not be aggregated with other data. He also said that users should not worry, yet, about specific metrics: "because each metric will need to be debated separately before it gets approved to be collected". He also said he did not see a strong reason to collect information about the country a user is located in.

Even though users have to opt into metrics collection, the proposal indicates that the yes/no toggle would have no default value: this means that users have to explicitly choose one or the other. Marc Deop i Argemí said that was unacceptable and the default position should be "off". "Most users will just click on 'Yes' without really comprehending what they are doing. And you _know_ this." Gary Buhrmaster replied that it was likely a matter of the phrasing of the prompt and wanted to know what the proposed wording would be.

Catanzaro said that they do not have the wording yet, but it would be something along the lines of "Help improve Fedora by sending anonymous usage data". The design of the question was critical, he said, "because if the acceptance rate is too low, then the project will fail and we're just going to be back here in a couple years" to debate making the metrics opt-out instead. Deop responded "you make it sound like you will never stop until you get the metrics you want".

FESCo votes

As of this writing, the proposal is not formally accepted but it is well on its way. According to FESCo ticket voting rules, if a change request receives at least three votes in favor and no votes against within one week, the change is approved without further discussion. If any FESCo member votes against the change, then it is added to the next FESCo meeting agenda for discussion and a majority vote would be required to accept the change. The change request has six votes in favor as of this writing, out of nine, and none against. Voting began on July 15, so the proposal will be approved on July 22 unless one of the remaining FESCo members votes against it. Even if a member objects and triggers a discussion, six votes for approval would easily clear the bar.

The next steps will be the formation of the metrics SIG and discussions about the final implementation of the system and metrics to be collected. Catanzaro has invited those who are interested in joining the SIG to contact him. "I hope to be wrong, but I expect it may be difficult to find people who are interested in participating." He later said that he had three volunteers, which was more than he expected, but still not enough.

There is still much work to be done, and discussion to be had, before any metrics will be collected. The current schedule for Fedora 42 has a deadline of January 14, 2025 for adding new features to Workstation, and February 25 as the deadline for removing any features that are not ready for the release. Final release is scheduled for mid-April 2025.



to post comments

They already obtained a metric ton of metrics

Posted Jul 22, 2024 16:16 UTC (Mon) by flussence (guest, #85566) [Link] (14 responses)

So, the Discourse forum helpfully tallies up that most of those threads got 100 individual respondents, and the straw poll got over 500. That's an incredible gold mine of users just volunteering personal data points about a distro they're invested in, quite a few with their name and mugshot attached, without any opt in necessary, and per the forum's existing privacy policy it's open season to use it all for research.

Whoever orchestrated all this ought to be getting a raise from IBM.

They already obtained a metric ton of metrics

Posted Jul 22, 2024 18:26 UTC (Mon) by zdzichu (subscriber, #17118) [Link] (3 responses)

Discord's userbase have zero value for distro development. It doesn't tell us what GNOME extension they use, what packages they use and so on.

I, for one, cannot wait until I can enable metrics on my systems. I want my use-cases to be taken into account in future Fedoras.

They already obtained a metric ton of metrics

Posted Jul 22, 2024 19:11 UTC (Mon) by tome (subscriber, #3171) [Link]

> I, for one, cannot wait until I can enable metrics on my systems. I want my use-cases to be taken into account in future Fedoras.

Same here. I also hope participation is robust in general, so I can make better guesses at how likely it is that the suits will continue to support the various aspects of the project, and to encourage investment by hardware partners.

They already obtained a metric ton of metrics

Posted Jul 23, 2024 8:21 UTC (Tue) by jccleaver (guest, #127418) [Link]

>I, for one, cannot wait until I can enable metrics on my systems. I want my use-cases to be taken into account in future Fedoras.

In theory, your use-case should be taken into account in future Fedora releases if you post or respond to the devel list.

In practice, a use-case that doesn't revolve around making someone's laptop work, shrinking a container, or finding a way to move more code into systemd is unlikely to get much traction. I wouldn't expect telemetry to change this too much. And as the telemetry's not any sort of especially representative sample of the user base, it probably *shouldn't* carry much weight other to than to confirm that somebody out there is, in fact, doing thing XYZ or is running on ABC.

They already obtained a metric ton of metrics

Posted Jul 24, 2024 5:18 UTC (Wed) by LtWorf (subscriber, #124958) [Link]

Or it will be like "only one user does X, we can safely ignore the use case".

They already obtained a metric ton of metrics

Posted Jul 22, 2024 20:40 UTC (Mon) by mattdm (subscriber, #18) [Link] (5 responses)

> Whoever orchestrated all this ought to be getting a raise from IBM.

It's really kind of funny to see the kinds of things people imagine IBM cares about from Red Hat.

In actual fact, IBM isn't involved in any of this, and doesn't actually have anything to do with the compensation / raise / bonus process at Red Hat.

I know conspiracy theories are fun and all, but it gets tiresome.

They already obtained a metric ton of metrics

Posted Jul 22, 2024 20:47 UTC (Mon) by mattdm (subscriber, #18) [Link]

That said, I am all for people who work on Fedora getting raises.

They already obtained a metric ton of metrics

Posted Jul 23, 2024 12:10 UTC (Tue) by ebiederm (subscriber, #35028) [Link] (3 responses)

That is an interesting defense, given that Red Hat only had profit sharing and not bonuses for engineering before IBM.

The experience of being acquired and whatever pressure IBM has placed on Red Hat has most definitely changed things.

I am still a bit in shock that the motto for the managers of a stable distro like RHEL became move fast and break things.

If Red Hat is involved I would not be surprised if some boss somewhere wants numbers they can cherry pick from to win a boss fight. Given the sheer amount of closed source software that was deployed inside RedHat after the acquisition I certainly would not expect Red Hat's management to have motives that are pure or in line with the greater open source and free software communities.

They already obtained a metric ton of metrics

Posted Jul 23, 2024 19:51 UTC (Tue) by mattdm (subscriber, #18) [Link]

> That is an interesting defense, given that Red Hat only had profit sharing and not bonuses for engineering before IBM.

This isn't correct.

> I am still a bit in shock that the motto for the managers of a stable distro like RHEL became move fast and break things.

This isn't about RHEL.

> If Red Hat is involved I would not be surprised if some boss somewhere wants numbers they can cherry pick from to win a boss fight

The actual specific "boss somewhere" is a human being, named in the 3rd paragraph of the article. In fact, it's this guy: https://blogs.gnome.org/uraeus/

They already obtained a metric ton of metrics

Posted Jul 24, 2024 0:31 UTC (Wed) by rfontana (subscriber, #52677) [Link]

> Given the sheer amount of closed source software that was deployed inside RedHat after the acquisition

Eh? This seems to imply that a result of the acquisition was that the amount of closed source software deployed inside Red Hat increased as a consequence of the acquisition. In reality, there was a fair amount of closed source software deployed inside Red Hat before the acquisition. If there was a material change during this time period (though still probably not proximately caused by the acquisition) it was probably a reduction of some on-premise proprietary software in connection with adoption of certain third-party SaaS offerings.

They already obtained a metric ton of metrics

Posted Jul 24, 2024 16:41 UTC (Wed) by smoogen (subscriber, #97) [Link]

> That is an interesting defense, given that Red Hat only had profit sharing and not bonuses for engineering before IBM.

I have worked for Red Hat on and off since 1997, and that has never been the case in any part of engineering I have been with. Bonuses might have been linked to 'profitable quarters' but that isn't profit sharing.

> Given the sheer amount of closed source software that was deployed inside RedHat after the acquisition I certainly would not expect Red Hat's management to have motives that are pure or in line with the greater open source and free software communities.

Red Hat has had large amounts of closed source software since I first joined. There have been times where the push was to make as much as possible Open Source but many of the things people complain about like Jira instead of bugzilla or Google Workplace were in the works way way before IBM purchased Red Hat.

They already obtained a metric ton of metrics

Posted Jul 22, 2024 20:53 UTC (Mon) by DanilaBerezin (guest, #168271) [Link] (3 responses)

> without any opt in necessary

Voluntarily posting on a public forum is by definition opt-in. What you've just described is basic OSINT and has been around since public channels of communication have been around. There is nothing inherently malicious or evil about it, it's just something you should always be cognizant of.

They already obtained a metric ton of metrics

Posted Jul 22, 2024 23:28 UTC (Mon) by atai (subscriber, #10977) [Link] (2 responses)

Open Source Intelligence... do you ask the Open Source Initiative if you use the term correctly?

They already obtained a metric ton of metrics

Posted Jul 23, 2024 8:00 UTC (Tue) by farnz (subscriber, #17727) [Link] (1 responses)

The term "open source intelligence" predates the Open Source Initiative by at least 8 years; the earliest use I can find of it in non-classified sources is in "Steele, R. (1990) ‘Intelligence in the 1990’s: Recasting National Security in a changing world’, in American Intelligence Journal, Summer/Fall 1990". And it may well have been used before then - Steele defines OSINT as "intelligence from open sources", implying that the concept of open sources in intelligence is something that readers should be familiar with.

They already obtained a metric ton of metrics

Posted Jul 23, 2024 12:16 UTC (Tue) by somlo (subscriber, #92421) [Link]

> The term "open source intelligence" predates the Open Source Initiative by at least 8 years

It's rather ironic, in retrospect. The OSI was conceived, IIRC, to make Free Software more palatable to for-profit business, and its founders didn't predict the "collision" with the pre-existing use of Open Source in military, intelligence, and law enforcement contexts.

I brought up Open Source (software) at a DoD workshop a few years back, and got a pretty adamant negative reaction from quite a few members of the audience, in whose mind "Open Source" was strongly associated with the concept of "monkeys on typewriters" and "things some rando was overheard saying in a bar" :)

And look at Free Software itself: constantly having to explain it's "free as in freedom, not free lunch". As any modern political type knows: in a debate, if you're finding yourself having to explain, you're losing.

Per-system or per-user consent?

Posted Jul 22, 2024 22:09 UTC (Mon) by Karellen (subscriber, #67644) [Link] (1 responses)

In addition to information about the systems that Fedora Workstation runs on, the working group is after details about how the system is used. For example, how applications are launched, how often users switch between windows and the methods used to do so, the number of applications open, and much more.

...

During initial setup, users will be asked if they consent to data collection. If users consent initially, they can disable metrics collection later in GNOME's system settings.

...

Of course, many users will be upgrading to Fedora 42 rather than doing a fresh install. For those systems, Neal Gompa said on the fedora-devel list that metrics will be off by default because there is no way to prompt the user to make the choice.

Wait, is the consent and data collection per-system, or per-user? It sounds like the proposal is for a purely per-system setting. That seems reasonable for collecting per-system data like hardware info, or which packages are installed, but it doesn't sound reasonable to me for per-user data like how often they switch windows or how many apps are open.

Surely each user should have to opt-in individually to have their usage data collected, for this system to make any sense. I don't see users can be considered to have consented to data collection otherwise.

(Yes, I know most workstations only have a single user. But not all will. And even for those that do, the single user is not necessarily the person who performs the initial install.)

Per-system or per-user consent?

Posted Jul 25, 2024 11:56 UTC (Thu) by wjt (subscriber, #56250) [Link]

The framework's on/off setting is per-system. Adding a per-user setting would be interesting indeed.

Counting how many systems are truly multi-user is actually one of the data points I (and I know others in GNOME) would like to see. I say “truly” because, although my own laptop has 3 accounts besides my own, most months they're totally unused. We never did quite finish that piece of instrumentation for Endless OS.

Against opt-in metrics

Posted Jul 22, 2024 22:17 UTC (Mon) by notriddle (subscriber, #130608) [Link] (18 responses)

Silently collected metrics are pretty hard to interpret into useful information (unlike surveys, for example, metrics can't ask what the user's intent is), but they make up for it because they get revealed preferences, instead of whatever the user thinks the interviewer wants to hear.

But you only get that upside if you have a statistically representative sample of the user population, and opt-in metrics aren't statistically representative.

Debian has opt-in metrics. It reports that 37% of machines regularly use xserver-xorg-core [^1]. That seems extremely unlikely to be a true reflection of their userbase, since general GNU/Linux usage share metrics shows lots of servers and very few desktops [^2]. Also, I suspect that popcon systemically excludes Docker containers that don't have cron.

By having opt-in metrics, you get all the downsides of metrics-driven development (information-poor data) without the upsides (accurate data). If you think "everybody lies," then opt-in isn't useful, because the only people who opt-in are the loud minority. If you trust your users to tell the truth, at least when it's in their own interest, then metrics are a lot less useful than surveys.

[^1]: Based on https://popcon.debian.org/by_vote by dividing vote count (86505) for xserver-xorg-core by the vote count for perl-base (219666): this seems like a good way to lowball how many popcon users have a GUI, since standalone xserver and xwayland both use xserver-xorg-core.

[^2]: Based on https://en.wikipedia.org/wiki/Usage_share_of_operating_systems#Market_share_by_category

Against opt-in metrics

Posted Jul 23, 2024 12:43 UTC (Tue) by ballombe (subscriber, #9523) [Link]

As popcon maintainer, I do not disagree with your analysis, but opt-out and even mandatory reporting are also biased (e.g. toward always-on / always-online system).

Against opt-in metrics

Posted Jul 23, 2024 12:57 UTC (Tue) by aragilar (subscriber, #122569) [Link] (15 responses)

I'm not sure you should trust opt-out metrics any more than opt-in, especially when a non-trivial subset of users opt out. I recall a blog post (https://chuttenblog.wordpress.com/2020/11/05/data-science...) from a data scientist from Mozilla who seeing that alsa wasn't commonly used on linux (according to their telemetry), required the use of pulseaudio. This turned out not to be the case (interestingly, based on some of the emails, this telemetry didn't even reach an actual release before the change happened), but to me the bit that stood out (both in the blog post and the linked mailing list thread) was the blame put on linux users and distros for not having telemetry on (which seems both tone deaf and a misunderstanding of the group they're targeting), and that even as this conversation is playing out, they acknowledge they had only tried contacting the Ubuntu maintainers (rather than using a more general channel to try to reach out more widely).

I suspect for both opt-in and opt-out, the best you can hope for is looking at trends over the "long term", and if this isn't something that has a relatively rapid turn around like a browser LTS (e.g. ~1 year changeover cycle) and instead lines up with linux distro LTS cycles, you'd probably need at least 6-7 years of data to be able to trust the data (e.g. if a new Fedora came with the discusses telemetry system this year, I don't think you could have a good handle on the results until 2026). Anything shorter you're likely missing a large amount of the population, no matter whether it's opt-out or opt-in (as Mozilla learnt above).

Against opt-in metrics

Posted Jul 23, 2024 18:25 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (13 responses)

> ...but to me the bit that stood out (both in the blog post and the linked mailing list thread) was the blame put on linux users and distros for not having telemetry on (which seems both tone deaf and a misunderstanding of the group they're targeting), and that even as this conversation is playing out, they acknowledge they had only tried contacting the Ubuntu maintainers (rather than using a more general channel to try to reach out more widely).

That's not how this works. Mozilla is under no obligation to contact every distro under the sun, or even the major distros, before making a change in their software. The amount of red tape that would imply is unreasonable. They have a system for voting for features, which is far more scalable than individual humans emailing each other - it's called telemetry, and if you turn it off, you are choosing not to vote. Of course, you are within your rights to do so, but you are not within your rights to then turn around and complain that your vote was not counted, when you never even cast one to begin with.

Against opt-in metrics

Posted Jul 23, 2024 20:04 UTC (Tue) by Wol (subscriber, #4433) [Link] (7 responses)

> That's not how this works. Mozilla is under no obligation to contact every distro under the sun, or even the major distros, before making a change in their software.

I think you need to read the following, which you apparently didn't before you replied ...

> > ...but to me the bit that stood out (both in the blog post and the linked mailing list thread) was the blame put on linux users and distros for not having telemetry on

Asking a notoriously privacy-conscious bunch to turn on a privacy-invading technology is bound to end in tears. Don't blame others if you can't be bothered to actually think through the - OBVIOUS - likely consequences of your actions.

It regularly blows my mind at work how managers assume our (minimum wage near enough) delivery drivers are stupid. From being one myself, A LOT of them are highly educated, A LOT of them quit the rat race. And the consequences of treating them like idiots is A LOT of money flushed down the pan. Managers are rarely as clever as the people they manage, but they like to think of themselves as superior. The best managers are good leaders, unfortunately the majority couldn't find their way out of a wet paper bag ...

Cheers,
Wol

Against opt-in metrics

Posted Jul 23, 2024 20:28 UTC (Tue) by pizza (subscriber, #46) [Link] (4 responses)

> It regularly blows my mind at work how managers assume our (minimum wage near enough) delivery drivers are stupid. From being one myself, A LOT of them are highly educated, A LOT of them quit the rat race. And the consequences of treating them like idiots is A LOT of money flushed down the pan.

Be careful, the plural of anecdote is not 'data'.

Against opt-in metrics

Posted Jul 24, 2024 9:18 UTC (Wed) by Wol (subscriber, #4433) [Link] (3 responses)

> Be careful, the plural of anecdote is not 'data'.

"The exception proves the rule" - used *correctly*, ie not how most people use it, it only takes ONE counter-example to destroy a scientific theory. And it's well known that people dismiss inconvenient facts because they can't face up to the fact their beliefs may be wrong. That's blatantly obvious in the world of tech and science right now ...

To you it may be an anecdote, to me it's *OBSERVED* *FACT* (sadly). I've had a manager say bluntly "drivers are too stupid to understand", meanwhile I've got an MSc in Medical Science, a guy I was chatting to used to be Head of Catering for British Airways - our National Airline, and I've chatted to many other - almost always older, true - guys with that sort of experience or qualifications.

Admittedly, we also have a lot of youngsters who are totally wet behind the ears and you really don't have a clue how on earth they managed to get their paper qualifications ... like the guy I was chatting to yesterday who just could not understand why it was a problem that our super-duper-new system would respond to timing challenges by "abandoning" vans and not tell anyone. FFS we're a supermarket! and to have us only finding out when 20 customers per van are ringing us up and asking us "Where's our shopping?" And he thinks that's not a problem!!! That's a LOT of pissed off customers!

Cheers,
Wol

Against opt-in metrics

Posted Jul 24, 2024 12:00 UTC (Wed) by pizza (subscriber, #46) [Link] (2 responses)

> To you it may be an anecdote, to me it's *OBSERVED* *FACT* (sadly).

Sure. But you are incorrectly making a *generalization* from one observation, and treating it as the norm.

Against opt-in metrics

Posted Jul 24, 2024 14:47 UTC (Wed) by Wol (subscriber, #4433) [Link] (1 responses)

But when Science says that's to be expected? And it's actually not one observation, it's a lifetime of observing.

It seems to be the case that nature hands out people skills and intelligence in inverse proportion to each other. Managers get to be managers because of their good people skills (not always true, the Peter Principle, anyone?). In other words, it's a pretty safe bet to assume that managers are, on the whole, a pretty stupid bunch. Almost invariably less clever than the people they manage (or if they're cleverer, they're usually incompetent managers). That's not slagging off managers, management is a whole different ball game that I would struggle with immensely - it's a completely different skill set. But being logical is usually NOT a skill set required or exhibited by management. They rely on us underlings for that.

But it's refreshing to find someone who (a) is a competent manager, and (b) is intelligent enough to understand what their underlings are doing. Precisely because it's unusual.

Cheers,
Wol

Against opt-in metrics

Posted Jul 24, 2024 16:04 UTC (Wed) by pizza (subscriber, #46) [Link]

> But when Science says that's to be expected? And it's actually not one observation, it's a lifetime of observing.

So? My "lifetime of experience" (or heck, even just the last two years of living of a dirt road in the middle of nowhere) is the opposite -- that most delivery folks can't reason themselves out of a wet paper sack. Or are even capable of reading delivery instructions printed on the slip.

Which "science" also supports in that more capable folks (and/or those with more options) typically find less physically taxing ways of making money.

Even if you accept that our individual observations are accurate, they represent far too small of a sample size to be considered statistically significant, much less representative of the population as a whole.

So while there may be MANY [emphasis yours] highly educated folks at your employer doing deliveries out of choice, MANY at one employer is a far cry from *most* at *every* employer.

Against opt-in metrics

Posted Jul 23, 2024 20:38 UTC (Tue) by pizza (subscriber, #46) [Link]

> Asking a notoriously privacy-conscious bunch to turn on a privacy-invading technology is bound to end in tears. Don't blame others if you can't be bothered to actually think through the - OBVIOUS - likely consequences of your actions.

...Tears? WTF?

And need I point out that the "OBVIOUS - likely consequences" of not participating in data collection about your use cases is... that you get effectively ignored. Which means that you can't get your way by merely shouting louder than anyone else.

Against opt-in metrics

Posted Jul 24, 2024 0:11 UTC (Wed) by NYKevin (subscriber, #129325) [Link]

> I think you need to read the following, which you apparently didn't before you replied ...

You... think I quoted something without reading it? That was exactly the part of the comment I was replying to in the first place.

Against opt-in metrics

Posted Jul 25, 2024 9:54 UTC (Thu) by aragilar (subscriber, #122569) [Link] (4 responses)

Naturally Mozilla does not need to contact distros about changes they make, as it is the distros don't need to contact Mozilla about changes they make (such as e.g. replacing Firefox as the default with Chromium, or less radically changes to build configurations and what features are enabled by default). However, one would assume collaboration and coordination would naturally make everyone's smoother. While Mozilla naturally has more good will associated with it that most groups, that amount of good will is not infinite, and wasting it only drives people away to alternatives.

In this specific case though, I'd argue there were multiple points where an alternative approach would have avoided this, and this whole situation was entirely preventable:

1. The reliance on telemetry over directly asking maintainers, where you could have had a shorter turnaround on your question, and had a more reliable answer. That is it from a process perspective easier to push opt-out telemetry (and then wait for it to propagate to users) rather than send a single email to maintainers says to me you process is broken (really, is "Hi All, We want to make Pulseaudio required, can you let us know by <date that's a week from this email> if this would cause problems for you" to a packagers/distros email list that hard to do)? The question of "What's the status of telemetry in Distro builds?" in less than 7 hours after the initial email suggesting the change. That there seemed to be no clear answer to this question should have been the first red flag. Note that it took more than a month for the telemetry results for alpha users to come it, which could have been spent getting replies back from the maintainers on what the majority of users were using.

2. Not waiting on the results to come in. The change was pushed based on alpha and beta users statistics, rather than waiting on users of the main release stream (and assuming that there would be no difference in the groups). It would seem to me if you're going to collect telemetry, then you should do it right, and not unnecessarily introduce bias into your decisions when you could have waited for a wider audience to send telemetry. Would this have made a difference, I don't know (as far as I can see no one commented what the results were), but that this wasn't discussed in the blog post seems like a major oversight.

3. Not changing strategy in the face of new data. Once the bug reports start coming in, the question that should have been asked is "why didn't we see this, and how can we ensure we have a good understanding of what's going on". Continuing to rely on telemetry when it has failed the first time should provoke a rethink, and the easiest way of getting alternate information would be to ask widely (you could e.g. post on social media, send out emails, IM people). Instead this seems to have never happened, with the blame put on distros and users. Given 2., it's not even obvious whether that would have made a difference, given distros are mostly likely to be packaging the release or LTS stream (which is apparently when this came in, so even worse), so what motivation do distros and users have to switch on telemetry, if it's just going to be ignored when convenient.

This took 9 months to flow from initial email to the first bug report, when a single email(!) could have meant that a coordinated change was made, resulting in less stress for all parties. I know which option I would choose.

Against opt-in metrics

Posted Jul 25, 2024 12:43 UTC (Thu) by pizza (subscriber, #46) [Link] (2 responses)

> 1. The reliance on telemetry over directly asking maintainers, where you could have had a shorter turnaround on your question, and had a more reliable answer

So, pray tell, how exactly are "maintainers" going to answer the question of "what plugins are your users installing?"
How will maintainers know about application crashes or non-crash runtime failures?
How will maintainers know which major features are used, and which are completely ignored?
How will maintainers know about the hardware capabilities of the install base?
How will maintainers know about... <fill in the blank>

The tl;dr is that maintainers *can't* answer these questions, unless ... they collect their own telemetry.

And your use of "pulseaudio" is complete nonsense, and is a perfect example of people intentionally choosing to do something different projecting their preferences onto to the majority. By the time Firefox ditched their native ALSA path, PulseAudio had been the default for *every* mainstream [1] Linux distro for literally years [2]. Any honest "maintainer" would have just answered "We don't know, and have no way of knowing this."

Folks insisting on using bare ALSA on modern systems were a rounding error of the rounding error that is Linux desktop market share. But they are a _very_ vocal rounding error that has been used to getting their way via SPAMing [3] loudly. Actual data collected via telemetry strips them of this power and sense of self-importance.

[1] Even Slackware, as it turns out.
[2] Even "native ALSA" applications were being routed through PulseAudio thanks to an alsa-lib plugin that redirected everything over to PA.
[3] in the sense of the Monty Python skit

Against opt-in metrics

Posted Jul 26, 2024 5:42 UTC (Fri) by aragilar (subscriber, #122569) [Link] (1 responses)

I would have though asking "are you building with this feature?" to be entirely answerable by maintainers? If the majority had said "no, we're not", then you have more information than if you simply got telemetry. And if you got responses of "yes, we are", you can ask *why* (the bit that telemetry doesn't get you)? If you deem those reasons insufficient, then you can still remove that feature, but you can also as the speaker noted manage the change and control the story. Naturally maintainers won't know everything, but again working with them may allow for better information to be collected (does keeping stack frames help with the bug reporter/error handler? asking that all the dependencies also have that configured correctly would seem to me to be better).

I have no issue with pulseaudio (though I've now migrated to pipewire, but that's because of new features rather than any deficiency on the pulseaudio side), and I wasn't affected by this change, but I do think as software engineers we are too quick to look at using technical solutions to social problems (whether that's driven by training, process or personality), and we should consider how we and our actions are perceived by society, given at least some groups (including Mozilla) want to public good.

Against opt-in metrics

Posted Jul 26, 2024 13:09 UTC (Fri) by pizza (subscriber, #46) [Link]

> I would have though asking "are you building with this feature?" to be entirely answerable by maintainers? If the majority had said "no, we're not"

If asked, pretty much every maintainer would have said "yes, we're enabling it", though some of them were actually wrong [1].

But the pertinent question isn't "is <feature> enabled", it's "how widely is <feature> _used_?" -- An honest distro packager can only reply one way -- "I don't know, and I can't know unless I add in telemetry."

Like it or not, there were significant technical issues [2] with Firefox's ALSA support that would have required a significant amount of work to (only partially!) resolve. They made a cost/benefit decision based on (a) their telemetry data, and (b) the platform defaults of *every* significant linux distro -- and easily came to the conclusion that "don't let the inherent limiations of our ALSA implementation keep us from improving the user experience for the vast, vast, vast majority of our userbase" was the best path forward.

Meanwhile, distros could have patched ALSA back into Firefox. Or those complaining could have stepped up and maintained said patches/builds or undertaken the work to address those technical problems. But either one would have required actual work, and wouldn't you know, work is haaaaard and must only be done by someone else.

[1] When Firefox first deprecated (ie not removed) ALSA, they switched the default from "build" to "don't build". Turns out many distro packages weren't specifically requesting ALSA support to be enabled, instead relying on upstream defaults.
[2] Interactions with multiprocess and sandboxing, audio exclusivity conflicts between multiple windows/tabs/other applications, etc, all leading to an objectively inferior user experience compared to literally every other platform (and competitor).

Against opt-in metrics

Posted Jul 25, 2024 12:53 UTC (Thu) by ballombe (subscriber, #9523) [Link]

> This took 9 months to flow from initial email to the first bug report, when a single email(!) could have meant that a coordinated change was made, resulting in less stress for all parties. I know which option I would choose.

Mike Hommey (from the Debian firefox packaging team) answered rather quickly, but he was ignored.
The "opt-in metrics" was one of the reason invoked to ignore him.

Against opt-in metrics

Posted Jul 24, 2024 2:03 UTC (Wed) by notriddle (subscriber, #130608) [Link]

> the bit that stood out was the blame put on linux users and distros for not having telemetry on

The actual blog post you linked to seems to place almost all the blame on distros, not users.

This makes sense, because while I would expect plenty of Linux users to opt out, the number isn’t the problem. The problem is the *skew*, caused by Arch turning it off while Ubuntu turned it on.

Against opt-in metrics

Posted Jul 25, 2024 14:00 UTC (Thu) by daenzer (subscriber, #7050) [Link]

> [^1]: Based on https://popcon.debian.org/by_vote by dividing vote count (86505) for xserver-xorg-core by the vote count for perl-base (219666): this seems like a good way to lowball how many popcon users have a GUI, since standalone xserver and xwayland both use xserver-xorg-core.

The xwayland package doesn't depend on xserver-xorg-core or use anything in it. Maybe you mean xserver-common?

(A Wayland session can work without Xwayland in principle, that's probably still negligible at this point though)


Copyright © 2024, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds