The Real AI Safety Problem Is Already Here

The existential threat of an AI doomsday is apparently all anyone can focus on right now. I believe the existential risks, while not impossible, are distracting from two far more important things that are currently happening. The first, which I talk about often, is that AI is causing real, serious harm TODAY. Scams, fraud, sextortion, deepfakes, harassment: all of these are happening right now, to real people, at a scale that would have been impossible even two years ago.

The second is a battle going on right now over the ability for labs to retain their customers’ data.

Labs claim they need access to a company’s data in order to identify and prevent abuse. But enterprises assume the labs are training on everything they see, as seen via the Millennium Prize, so they want ZERO data retention (“ZDR”). Is data retained under a safety carve-out being trained on? Probably not. But “probably not” is not good enough when proprietary data has become incredibly valuable and trust in the industry has completely eroded. Many companies are restricting model use today because of this fear. The exact visibility labs say they need for safety is the visibility enterprises refuse to give them because of data sovereignty concerns.

That’s why, in addition to rising costs, companies are moving to open-weight deployments: because those models are inherently ZDR. But that shift to open-weight models brings us back to the first problem: when you lose the lab’s safety infrastructure, you inherit the risk and are on your own. The labs know this; thus, the doomsday cycle continues. But it’s having the opposite effect: the louder the warnings get, the less enterprises trust the labs, the more they move to open-weight models, and the less safe the whole ecosystem becomes, which gives the labs something new to warn about.

Companies are being told to choose between privacy and safety. That shouldn’t have to be a choice.