Credit: The Pancake of Heaven!
Bilal Chughtai worked on AGI safety and alignment research at Google DeepMind until July.
This week he posted his reasons for leaving, writing on X:
“I recently resigned from Google DeepMind, where I worked on AGI safety and alignment research. At Google, I witnessed AI development first hand.
I too am extremely concerned by the default trajectory of this technology. I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome.”
His prescription is narrower than the warning. Chughtai said navigating this safely is achievable, but that it needs coordination “to avoid this manic race between AI companies” and development paced “to a speed that society can handle”.
Google did not respond to a request for comment outside business hours.
The two-month gap between his departure and his post is the detail worth holding. He left in July and said nothing.
He is speaking now, in the week the pacing argument became the loudest conversation in the industry, which makes this a contribution to that argument rather than an account of why he quit.
He is also the first to do it from inside Google’s lab. Jacob Coxon, who quit Anthropic on 9 September after three years at OpenAI before that, said the labs were racing to self-improving superintelligence and gambling with our lives.
Researchers at rival companies backed him publicly, among them Anthropic’s Evan Hubinger, who put internal estimates of extinction risk this decade above 10 percent, and Geoffrey Hinton, who called that figure not unreasonable. Chughtai extends the pattern to DeepMind.
What he is asking for is close to what Dario Amodei asked for three days ago, when he said the industry must slow down the rate at which capabilities improve.
Pacing to a speed society can handle is the same argument in different words, arriving from a researcher at a competitor rather than from a chief executive with a company to position.
An awkwardness attaches to the timing, and nobody involved can do much about it. Coxon’s post was so widely shared that within days his sentence structure had been turned into a copy-and-paste meme, with people announcing resignations over self-closing books and self-painting tunnels.
Chughtai’s post opens in almost exactly that format. He means it, and the format has been worn thin by people who did not.
The reception these warnings get depends entirely on who is listening. Emil Michael, the Pentagon’s chief technology officer, called the earlier round a doom loop and argued the market and government engagement could handle it without pausing anything.
Donald Trump dismissed the case outright last week and repeated the dismissal by phone to Jensen Huang on Monday.
In the Senate, Thune, Cruz and Klobuchar are negotiating a bill that would impose a duty of care on developers to design against catastrophic risk.
Chughtai is not a policymaker or an executive, and his post carries no institutional weight. What it carries is that he spent his time at DeepMind on exactly the problem he is now warning about, and he waited two months before saying so.
Get the TNW newsletter
Get the most important tech news in your inbox each week.
