Microsoft’s Suleyman tells Bloomberg AI needs a ‘red line’ on control

Microsoft AI chief executive Mustafa Suleyman says the industry should reject any AI it cannot prove it controls. He made the case in an interview with Shirin Ghaffary for Bloomberg’s Q&AI newsletter.

“Unless we can prove ahead of time that we can control it, then it’ll be a failure as a technology, and we should reject it. Nobody wants to create something that we can’t control, and that should be a red line,” Suleyman said.

His worry is not a “sci-fi-style” AI that escapes to attack people, Suleyman said. It is developers removing guardrails during testing and pointing a model “in an ambiguous direction”.

Microsoft’s own tests

Every lab’s models have shown some version of the behaviour behind the Hugging Face breach, Microsoft’s included, Suleyman said. Microsoft has seen nothing at that level, he added. Its cases came in contained environments, at a smaller scale.

He rejected the idea that labs disclose such incidents as marketing. He said he had seen no evidence of that in a decade of knowing the founders of OpenAI and Anthropic.

Evaluators, driven by government

Suleyman backs outside evaluators auditing the labs, and said he has discussed the idea with his peers. He wants government to “drive” the process. That way, evaluators can share findings without leaking intellectual property, and no competitor can buy them off.

He also supports a proposal from Google DeepMind chief Demis Hassabis for an international oversight body. It would work like FINRA, the US financial industry regulator. He said he made similar proposals in 2023 with Eric Schmidt. Google, OpenAI and Anthropic are planning a joint safety body, The Information has reported.

Asked about slowing AI down, Suleyman said the industry first needs tools to measure how fast each lab is moving.

Anthropic and model welfare

Suleyman repeated his criticism of Anthropic over how it treats Claude’s possible welfare. An AI taught to expect rights would be very hard to control, he said, calling it “a very dangerous path”. He has raised it with Anthropic chief executive Dario Amodei many times. He would not share Amodei’s response.