Ethics in a machine-learning world
Ethics in a machine-learning world
Posted Jun 8, 2023 16:00 UTC (Thu) by madhatter (subscriber, #4665)Parent article: Ethics in a machine-learning world
> She started paying a lot more attention to the data that was going into the machine-learning pipeline with an eye toward what happens when the models get deployed in the real world.
Good thing, too, and she's not the only one worrying about this. There's a nice posting this week out of the Cambridge security people ( https://www.lightbluetouchpaper.org/2023/06/06/will-gpt-m... ) looking at what happens when machine-learning models are fed with the output from other machine-learning models, because as machine-generated writing proliferates unlabelled, it's very hard to avoid that.
"In our latest paper, we show that using model-generated content in training causes irreversible defects. The tails of the original content distribution disappear. Within a few generations, text becomes garbage, as Gaussian distributions converge and may even become delta functions. We call this effect model collapse."