LWN: Comments on "Bias and ethical issues in machine-learning models"

Bias and ethical issues in machine-learning models

nix — Tue, 10 Sep 2019 15:11:46 +0000

> I was reading offline and couldn’t readily guess what twin cities were meant

Obviously Ul Qoma and Besźel, right? (Right?)

Bias and ethical issues in machine-learning models

robbe — Tue, 10 Sep 2019 12:38:38 +0000

I found that the article was also pretty USA-centric (or -biased, if you like). More so than LWN typically is.

For example, I was reading offline and couldn’t readily guess what twin cities were meant – and I am not well versed in how the mentioned universities typically earn money.

Bias and ethical issues in machine-learning models

ballombe — Mon, 09 Sep 2019 18:56:04 +0000

> Biases in ML are the subject of this amusing and interesting article :
https://thegradient.pub/nlps-clever-hans-moment-has-arrived/

What makes the article strange is that anybody that teach human students has encountered this issue (student guessing the correct answer to a test from lexical cue), but somehow it is new to machine learning researchers ?

On the other hand, this can gives a pseudo-objective measure of how much student test answers can be guessed from lexical cue!

Bias and ethical issues in machine-learning models

marcH — Sun, 08 Sep 2019 23:26:58 +0000

tl;dr: garbage in, garbage out.

So while AI is finally producing something useful after a few decades, there are apparently still a few jobs that need humans. These humans will likely be _assisted_ by AI more and more often.

As this example shows, working with more and more machines will require being comfortable with data, science and numbers more than ever before. Too bad climate change and energy are for instance in the US questions of party affiliation much more than question of... degrees Farenheits and BTUs. Let's put all the blame on non-decimal units for the failed education ;-)

Bias and ethical issues in machine-learning models

marcH — Sun, 08 Sep 2019 19:29:20 +0000

In such a "big typo" case I would not just strike the spurious sentence through but remove it completely. Strike through is useful to correct readers who previously internalized something wrong but logical enough to be believed. But here this is just noise I wasted a fair amount of time on trying to understand what correction had been made.

Somewhat OT: Bias and ethical issues in machine-learning models

marcH — Sun, 08 Sep 2019 19:23:38 +0000

Yes, "Murder" is better because it doesn't try to hide the actual ambiguity of the real-world rule like "Kill" tries to.

Bias and ethical issues in machine-learning models

kmweber — Thu, 05 Sep 2019 16:48:21 +0000

Mar Hicks's article "Hacking the Cis-tem" in IEEE Annals of the History of Computing a few months ago is an excellent account of algorithmic bias as a social/historical issue, if anyone's interested. Historians and other STS scholars are increasingly turning attention to these issues.

Bias and ethical issues in machine-learning models

madhatter — Thu, 05 Sep 2019 16:24:18 +0000

I'm glad somebody made that point. When I read in the original article

> Although there's a difference between personal medical information and routine retail sales like potato chips, Target was probably within reasonable ethical bounds in sending the pregnancy information.

my hackles went right up. That might be within reasonable ethical bounds in the US, but the EU is definitely moving away from the idea that retailers are entitled to make any use of data arising from customer transactions other than those involved in processing the actual transaction.

Bias and ethical issues in machine-learning models

dvdeug — Thu, 05 Sep 2019 05:40:32 +0000

One wonders how you'll feel when the AI selects against people with multisyllable names (because all their programmers were named Wang, Li or Chen) or people with words like "Republican" on their resume.

In any case, the law cares. If you're writing code that evaluates people, you may raising legal issues for your company if you don't try to control for biases in the legally protected categories.

Bias and ethical issues in machine-learning models

nix — Wed, 04 Sep 2019 13:57:23 +0000

In this case it might well be genuine bias: the proportion of women in the output was not equivalent to the proportion in the general population. i.e. actual mathematical bias. (One wonders if you'd call it 'political bullshit' if it came out with a large bias in the opposite direction...)

Bias and ethical issues in machine-learning models

edeloget — Wed, 04 Sep 2019 09:17:46 +0000

Does that mean that the book is biased?

(Ok, I'm leaving).

Bias and ethical issues in machine-learning models

nilsmeyer — Wed, 04 Sep 2019 06:29:18 +0000

And a lot of these times these algorithms are used in advertising, which isn't a particularly ethical pursuit either.

Bias and ethical issues in machine-learning models

q3cpma — Tue, 03 Sep 2019 16:17:34 +0000

>After looking at results and realizing that the model was biased against women, the department created another model after changing obvious gender markers such as pronouns and names. But the model was still biased
>But the model was still biased
"But the model wasn't aligning with our/my bias".

Why do we need this political bullshit disguised as truism (i.e. ML doesn't erase biases), by the way?

Bias and ethical issues in machine-learning models

nilsmeyer — Tue, 03 Sep 2019 15:41:33 +0000

> But it’s not like we can avoid that anyway - remember, the vast majority of these examples are not trained on objective data, like say exam results (which can be biased but obviously a lot less so than things which are just human choices)

If you look at Exams and IQ tests, there are some clear distributions that don't really inspire comfort. Many people who argue for (unconscious) biases existing would probably have huge problems with the uneven distributions here.

> You might not think that, for example, women’s low rate of participation in engineering jobs is a bias. I don’t agree, but it’s a position one could hold.

You would need to be able to prove a couple of things:
1. rates of participation aren't (roughly) equal due to bias.
2. (optional) that bias creates harmful outcomes
3. this can be fixed by properly tweaked machine learning algorhitms

I think that's a tall order, especially if you not only look at professions like engineering, there are some with even greater disparity, there are very few female plumbers or bricklayers, men are under-represented in healthcare. This all has to fit together somehow. Are there any jobs that have 50:50 parity between sexes?

Bias and ethical issues in machine-learning models

jake — Tue, 03 Sep 2019 13:45:40 +0000

> Did I miss something ?

No, we did. In a late-breaking edit, we eliminated one of the examples, but missed that it was referred to further on. Sigh ...

jake

Bias and ethical issues in machine-learning models

Paf — Tue, 03 Sep 2019 13:29:14 +0000

Well, sure? But it’s not like we can avoid that anyway - remember, the vast majority of these examples are not trained on objective data, like say exam results (which can be biased but obviously a lot less so than things which are just human choices) - they’re trained on stuff that was generated through human choices.

So, yes - we have to decide what we want these things to do, and that means thinking about what biases might be present in them. You might not think that, for example, women’s low rate of participation in engineering jobs is a bias. I don’t agree, but it’s a position one could hold.

But consider a more extreme example, let’s say a criminal sentencing system that was trained only on sentencing information from the American south from 1900 to 1950, and is intended to tell a judge what a “reasonable” sentence is based on this data. If the input includes racial descriptors, I think *most* people would agree it’s not going to generate reasonable sentences for black defendants compared to white defendants.

So what would you do with that? I mean, you’d presumably, idk, try to pick a different input set or blind it to racial information, etc, etc.

And so you’re altering the model to fit your view of the world - your bias, one might say.

From there on in, it’s just an argument about what the right view of the world is and what data sets and models are good enough/objective enough/whatever to generate trained systems that give “good” results.

So yeah, you can obviously wrap yourself around the axle looking for your desired results and just bend the model/inputs until it gives them... but it also seems clear that the choice of input data etc is hugely important. And now we’re in a *massive* grey area of moral choice.

We’re not modeling physics here. We’re asking about human choices, so bias is everywhere.

Bias and ethical issues in machine-learning models

farnz — Tue, 03 Sep 2019 13:14:58 +0000

It's slightly more subtle than that - it's "make sure your machine learning biases match the biases our society claims to want, not those that it actually has".

Bias and ethical issues in machine-learning models

clugstj — Tue, 03 Sep 2019 12:32:54 +0000

A lot of this article sounds like "Make sure your machine-learning biases match our societal biases".

Bias and ethical issues in machine-learning models

mimor — Tue, 03 Sep 2019 09:30:08 +0000

For those that are looking for a plethora of examples where machine-learning models went wrong, I can recommend the following read:
https://en.wikipedia.org/wiki/Weapons_of_Math_Destruction
The examples are mostly US based, so if you're looking for a EU or other continent-example, this is not the book for you.

Bias and ethical issues in machine-learning models

mimor — Tue, 03 Sep 2019 09:27:44 +0000

> just as a factory has a button any employee can push to stop the assembly line

FYI: This is called an Andon cord/button: https://en.wikipedia.org/wiki/Andon_(manufacturing)

Bias and ethical issues in machine-learning models

wazoox — Tue, 03 Sep 2019 09:22:05 +0000

Biases in ML are the subject of this amusing and interesting article : https://thegradient.pub/nlps-clever-hans-moment-has-arrived/

Bias and ethical issues in machine-learning models

laarmen — Tue, 03 Sep 2019 09:02:09 +0000

I'm confused by the sentence "This stage is where the two companies mentioned earlier parted ways on the question of offering customer rewards." in the section "Modeling targets and business problems". Which companies are we talking about here ? AFAICT the only companies mentioned before this are O'Reilly Media and Minne Analytics. Did I miss something ?

Bias and ethical issues in machine-learning models

nilsmeyer — Tue, 03 Sep 2019 08:45:21 +0000

I think part of the problems is that reality doesn't always align with our biases on how it should be.

Somewhat OT: Bias and ethical issues in machine-learning models

vadim — Tue, 03 Sep 2019 07:54:05 +0000

I would say it's a lot more ambiguous, actually.

"Kill" is a pretty clear term.

"Murder" on the other hand just stands for "whatever kinds of killing that we happen to disagree with", which can mean absolutely anything. It's almost as bad of a rule as "do the right thing".

Bias and ethical issues in machine-learning models

scientes — Tue, 03 Sep 2019 01:14:39 +0000

...goes back to right swiping.

Bias and ethical issues in machine-learning models

scientes — Tue, 03 Sep 2019 01:14:05 +0000

I, personally, find that having us humans be bred by machines is a relief.

Somewhat OT: Bias and ethical issues in machine-learning models

dskoll — Tue, 03 Sep 2019 00:38:24 +0000

"Thou shalt not kill" is a bad translation. The original text is more like "Do not murder" which is a little less ambiguous.