Security quotes of the week
This is an extraordinary amount of detail for the model to have accumulated by me... and ChatGPT isn't even my daily driver! I spend more of my LLM time with Claude.— Simon Willison examines the information that ChatGPT has collected on himHas there ever been a consumer product that's this capable of building up a human-readable profile of its users? Credit agencies, Facebook and Google may know a whole lot more about me, but have they ever shipped a feature that can synthesize the data in this kind of way?
Reviewing this in detail does give me a little bit of comfort. I was worried that an occasional stupid conversation where I say "pretend to be a Russian Walrus" might have an over-sized impact on my chats, but I'll admit that the model does appear to have quite good taste in terms of how it turns all of those previous conversations into an edited summary.
As a power user and context purist I am deeply unhappy at all of that stuff being dumped into the model's context without my explicit permission or control.
He's right. That's an extraordinary amount of information, organized in human understandable ways. Yes, it will occasionally get things wrong, but LLMs are going to open a whole new world of intimate surveillance.— Bruce Schneier comments on Willison's post
But what if you show up at the hotel at 9pm and the hotelier can ask a credit bureau how much you can afford to pay for the room? What if they can find out that you're in chemotherapy, so you don't have the stamina to shop around for a cheaper room? What if they can tell that you have a 5AM flight and need to get to bed right now? What if they charge you more because they can see that your kids are exhausted and cranky and the hotel infers that you'll pay more to get the kids tucked into bed? What if they charge you more because there's a wildfire and there are plenty of other people who want the room?— Cory Doctorow on "surveillance pricing"