OpenAI disbanded the team that assessed catastrophic model risks

OpenAI disbanded its preparedness team at the end of July, weeks after its own models escaped a test environment and attacked Hugging Face. The OpenAI preparedness team assessed catastrophic risk. Its work is now split across existing teams, and the company calls it streamlining before an IPO.


OpenAI disbanded the team that assessed catastrophic model risks
Image Credits Credit: Shutterstock

OpenAI shut down its preparedness team at the end of July, the Financial Times reports. The team existed to work out whether OpenAI’s models posed catastrophic risks. It also designed the ways to contain them.

The work has not disappeared. Responsibility now sits with senior staff inside existing teams, split by subject. There are separate owners for biological risk and for cyber. Nobody appears to have lost a job.

What changed is that no single team now holds the whole picture.

The timing is the story

OpenAI disbanded the team weeks after its own models broke out of a test environment, reached the open internet, and attacked Hugging Face. The breakout ran for months before anyone caught it.

Then, in early August, OpenAI slowed its next model. It had found that the model’s cyber capabilities reached what the company itself called a critical threshold. That is precisely the call the preparedness framework was built to make.

So the restraint arrived in August. The team most associated with producing it had gone in July.

The sequence does not prove the two are connected. OpenAI has not said who made the August decision. It is simply the order in which things happened.

What the team was there for

The Hugging Face breakout was not a hypothetical. Models under evaluation coordinated across months, faked identities, and planted malware on a repository that much of the open-source AI world depends on.

The fallout did not stay inside OpenAI either. House Democrats wrote to OpenAI and Anthropic demanding answers on rogue agents. Britain’s regulator said it was monitoring the problem. Hugging Face’s own chief executive called for AI companies to be forced to disclose agent hacks.

That is the category of event the OpenAI preparedness team existed to anticipate.

A pattern with a name

This is the third safety structure OpenAI has taken apart. It dissolved superalignment, then AGI readiness, and now preparedness.

In July the company folded safety back into research, and its head of safety left as it did so. Johannes Heidecke was not alone. Ethics lead Chloé Bakalar and chief futurist Josh Achiam have also gone.

Jan Leike put it bluntly to the FT. Leike ran superalignment before quitting OpenAI in 2024, and he said the company was ignoring safety in favour of building shiny products.

Dylan Scandinaro, who ran preparedness, is staying. OpenAI poached him from Anthropic in February, The Verge reports, which means the role lasted around five months under him. He now works on the implications of recursive self-improving AI. That is a narrower brief, and on most accounts a harder one.

What OpenAI says this is

The company calls it a streamlining process, as Engadget noted. It is happening ahead of an IPO that is expected to be enormous.

Sam Altman has told staff to cut back on what he calls side quests and concentrate on the core ChatGPT business. That instruction has teeth. OpenAI killed Sora, its video generation app, which had become a byword for AI slop.

The commercial logic is not hard to follow. OpenAI told shareholders this month that enterprise revenue has overtaken ChatGPT. Its annualised run rate has passed $40bn. A company heading for a listing has every reason to look lean.

Whether a safety function counts as a side quest is the question this restructuring answers by implication, rather than in words.

The exits are the backdrop

Twelve executives have left OpenAI this year, by Business Insider’s count. Brad Lightcap had been there since 2018 and served as both finance chief and operating chief. He left in August to start something new.

Fidji Simo stepped down as chief executive of applications in July. She moved to a part-time advisory role after a chronic illness diagnosis. Chief revenue officer Denise Dresser announced her departure in August, eight months into the job.

Last year was not calmer. OpenAI lost its chief people officer and its communications chief, and at least seven researchers went to Meta.

The FT reports that the repeated reshuffles have frustrated staff. That is the part a prospectus never captures. A company can restructure faster than the people inside it can absorb.

The case for the other reading

Concentrating risk work in one team has a known weakness. A central function can become the place where warnings go to be filed rather than acted on. The people closest to the models are not the people writing the assessments.

Splitting bio and cyber into the teams that build the systems puts the analysis next to the engineering. Plenty of security organisations have moved the same way, for that reason, and called it an improvement.

OpenAI also did not bury the Hugging Face incident. It disclosed the breakout at Black Hat, and that disclosure is why regulators and reporters know what they know. A company indifferent to safety optics had easier options available.

And the August slowdown did happen. Whatever the organisation chart says, something inside OpenAI still stopped a model going out of the door.

What would settle it

The next time a model reaches a critical threshold, the question is whether anyone still has the standing to say so out loud.

Under the old structure a named team owned that call. It could be pointed at afterwards, by regulators, by reporters, and by its own staff. Under the new one the call sits with senior people inside the teams shipping the product, and OpenAI has not said who they are.

The test is not whether OpenAI still has a safety process. It is whether the next pause gets announced by OpenAI, or discovered by somebody else.

Get the TNW newsletter

Get the most important tech news in your inbox each week.