LWN: Comments on "Ethics in a machine-learning world"

Ethics in a machine-learning world

madhatter — Thu, 08 Jun 2023 16:00:18 +0000

> She started paying a lot more attention to the data that was going into the machine-learning pipeline with an eye toward what happens when the models get deployed in the real world.

Good thing, too, and she's not the only one worrying about this. There's a nice posting this week out of the Cambridge security people ( https://www.lightbluetouchpaper.org/2023/06/06/will-gpt-m... ) looking at what happens when machine-learning models are fed with the output from other machine-learning models, because as machine-generated writing proliferates unlabelled, it's very hard to avoid that.

"In our latest paper, we show that using model-generated content in training causes irreversible defects. The tails of the original content distribution disappear. Within a few generations, text becomes garbage, as Gaussian distributions converge and may even become delta functions. We call this effect model collapse."

Ethics in a machine-learning world

flussence — Tue, 06 Jun 2023 18:24:28 +0000

> She said that the current crop of chat bots (e.g. ChatGPT) are not actually particularly useful for search, despite what Microsoft and Google would have us believe; they are useful for other things, such as creative writing or learning English, but not search. Those companies are effectively paying for the models via their search tools, so they confirm their own beliefs even though it is not actually true, she said.

They aren't doing this to improve search, that would run counter to the core ideals Microsoft and Google embody of ruthless exploitation of the commons for profit (see also: a few paragraphs below that one). The system as-is doesn't produce profit either - only constant jokes about how bad it is. That's free mindshare.

They are *going to* make it turn a profit the only way unimaginative brainwormed tech managers know how. Look for the headlines of mass layoffs of frontline support workers in these corporations a few months from now. Using software and services from them already feels like an unwanted Milgram Experiment, and it's about to get a whole lot worse.

Ethics in a machine-learning world

jake — Tue, 06 Jun 2023 17:32:07 +0000

> The video of the talk is online.

Thanks for that link! I looked for it a few days ago, presumably just before the talk videos went up. The PyCon 2023 playlist at YouTube is here: https://www.youtube.com/playlist?list=PL2Uw4_HvXqvY2zhJ9A...

jake

Ethics in a machine-learning world

antacon — Tue, 06 Jun 2023 16:22:43 +0000

This was a great overview of the talk. Mitchell made worthwhile points to bring up when I inevitable debate the current "A.I." hype with a cohort. Thank you!

Ethics in a machine-learning world

Karellen — Tue, 06 Jun 2023 16:04:22 +0000

The video of the talk is online. It's 42:28 long, but I've not had a chance to watch it yet:

Keynote Speaker - Margaret Mitchell.