The TAB report on the UMN affair [LWN.net]

The TAB report on the UMN affair

Posted May 6, 2021 5:00 UTC (Thu) by epa (subscriber, #39769)
In reply to: The TAB report on the UMN affair by gus3
Parent article: The TAB report on the UMN affair

Aw c’mon, the Nuremberg Code? Does anyone take seriously the idea this was ‘experimenting on humans’, as if they’d dropped a new drug into Greg’s coffee without asking him? That seems like manufactured outrage to me. If you see the world like that, everyone who posts a trollish comment on LWN to see the reaction would need approval from an ethics committee. Retailers wouldn’t be able to vary their prices to see the effect on sales. Even ordinary patch submission (without the deliberate bugs) would count as an experiment on humans if later reported in a paper.

The TAB report on the UMN affair

Posted May 6, 2021 9:51 UTC (Thu) by rmayr (subscriber, #16880) [Link] (7 responses)

[Full disclosure: I am one of the four researchers who originally raised the concern to IEEE S&P chairs in November.]

Oh, yes, this is very much considered to have been intentional research on human subjects. It doesn't only require (potential) bodily harm, but any harmful effect that can be caused by an experiment without consent - wasted time included - is unethical. By this definition, experimenting on consumers with prices is also not on the positive side of an ethical debate, though many businesses operate that way right now. In this particular case, the research was not only intentional, but intentionally deceiving, which is a step up from neutral changes to watch for an effect.

However, the important part here is that, in pretty much all democratic/liberal countries with universities funded by public money, academic research is held to a much higher standard than private businesses. Research on human subjects requires their explicit, informed consent or, in *very* limited exceptions where that consent would undermine the research goal that is in the overarching public interest, close oversight by an independent committee. An academic research group can absolutely not decide by themselves if their human subjects experiments are ethical or not, and which safeguards to put in place.

The TAB report on the UMN affair

Posted May 6, 2021 10:34 UTC (Thu) by epa (subscriber, #39769) [Link] (2 responses)

I agree that academic research is held to a higher standard and I don't doubt the researchers might be in trouble with their ethics committee (or the committee will be in trouble for having issued a waiver). What I find odd is people who have no connection to the university or academia sticking their noses in and denouncing the researchers for perceived violations of some ethics code. Surely that's a sideshow. After all the intentionally broken commits could equally have come from a private individual or even someone working undercover for an intelligence agency.

Thanks for the clarification that it is indeed considered research on a human subject. I think it is a mistake to group this kind of tail-tweaking with real nonconsensual experiments forbidden by the Nuremberg Code (which very clearly is talking about medical experimentation). But then, I'm not part of the ethics committee either, so I'm not really qualified to comment.

The TAB report on the UMN affair

Posted May 8, 2021 17:39 UTC (Sat) by NYKevin (subscriber, #129325) [Link] (1 responses)

> After all the intentionally broken commits could equally have come from a private individual or even someone working undercover for an intelligence agency.

Such a person would have been banned from submitting patches and that would have been the end of it. Indeed, that's *precisely* what happened in this case, except that everybody decided that "UMN banned" is news, whereas "John Smith banned" is not news.

The TAB report on the UMN affair

Posted May 11, 2021 6:15 UTC (Tue) by epa (subscriber, #39769) [Link]

I'm saying that intentionally broken patches could have come from a bad actor who didn't go on to disclose that they were bad and publish a paper about it. Most likely they would never be spotted.

The TAB report on the UMN affair

Posted May 7, 2021 2:18 UTC (Fri) by tytso (✭ supporter ✭, #9993) [Link]

UMN, even in their most recent response, has claimed that it isn't considered Human Subject Research, so they disagree with you. I think they are full of sh*t, but they almost had to make that claim, given that the Hypocrite Commit work was funded by an NSF grant, and if they admitted that it was subject to the HSR rules, then (a) their IRB would probably in deep doodoo, and (b) they might have to refund their grant money to the NSF, and/or be subject to various disciplinary actions from the NSF. It's my understanding that a complaint has reached the NSF, and it'll be interesting to see what the good folks at NSF think of UMN's claim of, "no HSR work here"!

The TAB report on the UMN affair

Posted May 7, 2021 16:18 UTC (Fri) by nedu (guest, #50951) [Link] (2 responses)

> [Full disclosure: I am one of the four researchers who originally raised the concern to IEEE S&P chairs in November.]

You wrote "November" here, but in the TAB report, I'm seeing a Dec 1 event in the "Timeline of events".

| 2020 Dec 1:
| - Sarah Jamie Lewis & others send a letter to IEEESSP.
| https://hackmd.io/s/BJGs6Tfiw

What looks like an email published at that url seems to be undated.

Metadata in the source of that webpage seems to support a December 1, 2020 publication date. Or, simply hovering over /changed 5 months ago/ results in a tooltip.

Anyhow, this event in the TAB report's timeline is what you're referring to?

[Yesterday, I sent you an email asking about this.]

The TAB report on the UMN affair

Posted May 7, 2021 16:45 UTC (Fri) by deater (subscriber, #11746) [Link] (1 responses)

it depends if you count tweeting at the paper authors "reporting"

here's discussion of the issue in November
https://twitter.com/SarahJamieLewis/status/13306189193762...
after the paper authors had deleted the original tweet.

The TAB report on the UMN affair

Posted May 7, 2021 19:27 UTC (Fri) by nedu (guest, #50951) [Link]

As it turns out, I received an email from Rene this morning (7 May 2021 08:25:25 +0000), but entirely due to my own fault, I hadn't yet seen that reply when I posted my comment about 8 hours later.

Nevertheless, I do hope Rene takes the opportunity to discuss these late November thru first of December events here.

The email referenced in the TAB report itself contains a link to an archived Twitter exchange from 21 - 22 November 2020.

https://web.archive.org/web/20201122173246/https://twitte...

Please do scroll up to see the beginning of that Twitter exchange -- although I'm interested in discussing Kangjie Lu's tweet at the bottom, where he says, among other things:

> The paper will be available soon. [...] I can share a copy with you in email.

The TAB report on the UMN affair

Posted May 6, 2021 11:25 UTC (Thu) by Homer512 (subscriber, #85295) [Link]

I feel like even beyond any questions of ethics, kernel devs just don't need to tolerate this kind of behavior. I mean, we have a whole code of conduct which can be summed up as "Don't be an asshole."

The report mentions the case where a reviewer wasted their time trying to mentor the bad-faith contributor. In my opinion, the researchers should compensate the reviewers for their time spent.
The same goes for the last set of patches which were in good faith but so crappy that they were indistinguishable from bad faith acting. The reviewers are not beta-testers for these people's research projects. This isn't some newbie dev who needs a bit of mentoring. This is a whole research group that should have internal procedures and reviews before code leaves their department.

If the UMN acts in a way that wastes more dev time than it contributes, the kernel devs don't need to tolerate UMN devs in their community.

YES people take this seriously

Posted May 8, 2021 16:30 UTC (Sat) by david.a.wheeler (subscriber, #72896) [Link] (2 responses)

Short answer, YES. People DO take this seriously. You may not care if you're experimented on without your consent, but other people do care very much. As soon as you do experiments in the US, there are a number of rules and guidelines that are required, once humans are subjects. It can be drugs, it can be behavioral research, whatever, it doesn't matter.

The key in the US is the The Belmont Report: Ethical Principles and Guidelines for Protection of Human Subjects of Biomedical and Behavioral Research (1979), which says, “Respect for persons requires that [experimental] subjects... be given the opportunity to choose what shall or shall not happen to them… the importance of informed consent is unquestioned... the consent process [contains] information, comprehension and voluntariness [and generally includes the opportunity to] withdraw at any time from the research.” The Belmont Report is widely cited in the US as an ethical framework, it's the basis for the "Common Rule" required by US government agencies for federally-funded research.

A follow-on report, the Menlo Report (2012), was published by the U.S. Department of Homeland Security Science & Technology Directorate, Cyber Security Division, and outlines an ethical framework specifically for research involving Information and Communications Technologies (ICT). The Menlo Report adapted the original Belmont Report principles (Respect for Persons, Beneficence, and Justice) to the context of cybersecurity research & development, as well as adding a fourth principle, "Respect for Law and Public Interest." A companion report to the Menlo report provides case studies. Note that the Menlo report, since it built on the Belmont Report, also strongly emphasized the need for informed consent.

The IEEE released a statement that the paper "did not follow [ethical] guidelines". That's pretty harsh stuff in this space.

The good news is that UMN has agreed that this was a mistake. The paper's been withdrawn, and they're working to prevent recurrence. So while this affair was unfortunate, I think it's on its road to resolution.

YES people take this seriously

Posted May 11, 2021 6:24 UTC (Tue) by epa (subscriber, #39769) [Link] (1 responses)

I very much care about being experimented on. I'm just questioning whether a duff patch submission, or an April Fool's joke, or an advertisement placed in the press to see who's interested, are really on the same ethical scale as actual "experiments on humans" involving medicines, or surgery, or real world deception. By conflating the two and and trying to escalate this rather trivial timewasting into a serious ethics breach, I think people are trivializing the very serious matters that the Nuremberg Code, Belmont Report and so on are addressing.

I think penetration testing and test social engineering attacks are fairly common practice, and they don't have the consent of those who are being tested, not of the individual employees at least. My workplace regularly sends deliberately false messages as a phishing test to see who clicks on them. It's annoying, but I would not try to place it on the same ethical scale as administering drugs to employees without their consent, or deliberately depriving them of sleep to see what happens.

YES people take this seriously

Posted May 11, 2021 6:56 UTC (Tue) by amacater (subscriber, #790) [Link]

If your workplace regularly sends fake phishing emails to see who clicks:
1. It will have been signed off by somebody senior at some stage that this is appropriate.
2. It will be an authorised exercise.
3. It's probably a condition of employment that, if you're at work/using work systems then you are taken to have consented to terms of appropriate use of the system. You probably have signed up to terms of use at some point/had them pointed out to you.
4. That set of terms will permit login/security monitoring if appropriate.

If 1 and 2 don't apply, then somebody else is in breach of 3 and 4 :) If none of these
apply, you're in a similar position to the kernel devs. here.

The UMN researchers may have been unknowing/careless at best: devious and exploitative at worst. Their IRB may have been on the ball and questioned everything they saw before allowing it or they may have been unsighted/misled/not understood the scope of the work. The combined effect was that their actions impacted a bunch of third party developers, caused work, created a degree of mayhem. That's not OK.

A bunch of far smarter people than I am can argue the exact cost and harm but it's left a sour taste in the mouth for major kernel developers who are the people I rely on to provide me reliability and security every time I start my machine.