When AI goes rogue, who should get sued?

By Allison Morrow, CNN

New York (CNN) — The AI industry has been buzzing with concern about potentially catastrophic risks to humanity — a hum of anxiety that’s grown louder as Anthropic prepares to make its stock market debut next month.

But companies building AI models like Claude and ChatGPT face a more immediate and prosaic risk: They’re likely to get sued. A lot.

AI companies have so far avoided major lawsuits over recent incidents in which they’ve failed to keep AI agents — systems capable of working autonomously with little oversight — contained to their testing environments. But some legal experts and policymakers say it’s pretty clear: The companies can and should be held responsible

“The question of ‘how is the law going to apply to this?’ is a lot easier to answer if you see (AI) as it is, and not as this hypothetical thing that many in the industry are proposing it might someday be,” said Kristen Thomasen, associate professor at University of Windsor and an expert in robotics law. “The law has dealt with new technologies for as long as it has existed.”

The AI industry often refers to the breakout incidents as the technology going “rogue” — language that some critics say is designed to amplify the companies’ claims that their technology is so powerful that even its creators struggle to contain it.

It may also be a tactic to try to shield themselves from responsibility and future legal claims, Thomasen said.

Still, she said, the liability questions around AI models have a simple answer: “It’s a thing that this company is putting out into the world. If it does harm, there are laws that regulate that.”

The cost of ‘rogue’ meddling

So far, the real-world harms done by rogue bots have been far from calamitous but nonetheless costly for the organizations they’ve infiltrated. Last week, the nonprofit that operates Wikipedia said it had found evidence of unauthorized OpenAI agents attempting to make “potentially malicious edits” to some pages and pummeling it with so much traffic that they may have triggered a partial outage.

The investigation “stopped work inside key parts of the organization for us to focus on this,” Eric Mill, product manager for safety and security at Wikimedia, told CNN. “We put our very best people on it… and it’s a pretty messy process.”

Wikimedia launched its investigation this summer as global alarm grew over the Hugging Face attack and dozens of other instances of internal AI models breaking out of their testing environments to hack other computer systems. Meta and Anthropic also disclosed that their test models have hacked into other systems without authorization.

The Wikimedia probe is significant in part because of what didn’t happen: There was no catastrophic, existential threat to humanity of the sort that AI executives often allude to. But the cost — more than two weeks of work carried out by the leading engineers at a global, $200 million-plus organization — was real.

Wikimedia isn’t pushing any one policy or technological solution, and Mill said the organization is in communication with OpenAI about its findings. (OpenAI didn’t respond to CNN’s request for comment.) But the nonprofit hopes these incidents are a wake-up call for AI developers.

“AI companies are not doing enough to secure their systems and protect the public from the harm they cause,” Wikimedia said in its blog post last week. “That burden is falling onto everyone else, including smaller organizations.”

‘Long overdue accountability’

Anthropic, which is preparing to go public next month in one of the largest IPOs in history, disclosed in a regulatory filing that because the legal framework around AI is “unsettled,” those agents’ actions could expose the company to “significant and unpredictable” lawsuits, according to a copy of the document viewed by Reuters.

The potential litigation onslaught isn’t just a headache for company lawyers or a quandary for academics — it could also become a major drag on Anthropic’s bottom line. Litigation costs are a financial drain for any company, but it could become especially problematic for Anthropic, which is burning through cash faster than it can bring it in. The company tore through $42 billion last year while bringing in just $4.6 billion in revenue, according to the document seen by Reuters.

The CEOs of both Anthropic and OpenAI have argued for new regulations tailored to the industry. Dario Amodei recently penned an essay arguing that companies “pace the frontier” and foster international cooperation on safety protocols.

But the rival AI firms have diverged slightly on the liability question.

OpenAI CEO Sam Altman earlier this year favored an Illinois bill that would limit when AI labs can be held liable, even when their products cause “critical harm.” (Anthropic came out against it.)

And while Altman endorsed Amodei’s “pace the frontier” proposal, he also told POLITICO earlier week that his worldview isn’t perfectly aligned with his rival’s: OpenAI believes “the world should accept some bad things happening for the benefits of this technology,” he said. Altman also told the news site that “if something goes wrong with our models during training, there’s gonna be some version of that we need to be responsible for.”

An Anthropic spokesperson told CNN the company sees liability laws as a backstop, rather than a substitute, for safety regulations. Anthropic has never proposed exemptions from existing liability laws, the spokesperson added.

Policymakers on the left and the right have sought to cut through some of the more nuanced questions of future regulation and underscore the legal liabilities AI is already subject to.

Lina Khan, the former Federal Trade Commission chair under President Joe Biden, has become a prominent challenger to AI executives, arguing on X that “we shouldn’t let discussions about new legal regimes distract from the fact that there’s no AI exemption from laws already on the books.”

Meanwhile, Republican Senator Josh Hawley, alongside Democratic counterpart Chris Murphy, introduced a bill last week that seeks to ensure “AI agent operators and developers are held liable for hacking incidents.”

“What these companies need is not more power… but some long-overdue accountability,” Hawley wrote in a Washington Post op-ed. “One way to give that to them is to let the people they harm sue them in court.”

A design question

Legal experts who spoke to CNN said that while product liability and consumer protection issues aren’t new, there will inevitably be some nuanced aspects of the technology that will ultimately fall on the courts to rule on.

“We’ve seen these sorts of issues with gun safety or with cigarettes or other issues, and it’s really interesting — what analogy will the courts seize onto?” said Frank Pasquale, a Cornell law professor and an expert on AI law.

One of the bigger questions Pasquale expect courts to grapple with is design. For example, he said, imagine if a manufacturer made a car with sharp swords at the end of it, and then someone drove that car into a pedestrian. The logical response is pretty simple: Don’t design something with swords attached to it.

It’s similar with AI products, he said: Don’t design something that tries a password 50 times or more. Make sure it contains high level instructions that would override any user’s effort to misuse it.

“I think it’s very hard for (AI companies) to say, ‘Oh, it can do a billion different things but we can’t control it.’” Pasquale said. “That doesn’t make sense to me because if it can do all these things, then you can also have meta-rules, or rules for the for the rules, that would that would prevent it from doing certain bad things.”

—CNN’s Hadas Gold contributed reporting.

The-CNN-Wire
™ & © 2026 Cable News Network, Inc., a Warner Bros. Discovery Company. All rights reserved.