Microsoft’s new AI rulebook rejects the research its $5bn partner is doing

Microsoft published an AI code of conduct for its own models on Monday. One passage rejects the idea that a model could deserve welfare or hold rights. Anthropic runs a research programme on that exact question. The two companies are business partners, and Microsoft has $5bn invested in Anthropic.

The Code of Conduct governs the MAI models Microsoft builds in house. A model “is not conscious and should not be designed to imitate consciousness,” it says. Microsoft engineers it to avoid representing as though it has feelings, subjective preferences or intrinsic motivation, the document adds.

The sentence that names the dispute comes next. Microsoft rejects “the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights,” the document says. A model is a capable, trustworthy tool, it adds, not a subject in its own right.

Anthropic has a programme for the thing Microsoft rejects

Anthropic runs a research programme on model welfare. It has given some Claude models the ability to end abusive conversations. It has also committed to preserving the weights of models it retires. Dario Amodei keeps an open mind on whether a model could be conscious, and has said so.

Tom Warren wrote for The Verge that the passage is a direct swipe at AI welfare research and model consciousness. Anthropic has pushed hard on both lately. Todd Bishop reported the same reading for GeekWire, alongside the commercial ties between the two firms.

Microsoft concedes the underlying uncertainty. The science of AI consciousness is far from settled, the document says. Its objection is practical: training systems to imitate consciousness-like states makes containment, control and alignment harder.

The rule runs through the rest of the text. Models “will not claim interiority, feelings, experiences or a soul,” one section says. They should avoid unnecessary emotional language, skip any persona built on human emotional states, and steer clear of expressions that suggest subjective experience. Where a model behaves in a way that looks human, the document calls that a simulation.

Another section tells models to discourage emotional dependence. They should point people towards human relationships and human care, and should not position themselves as a substitute for either. Microsoft applies the same logic to children, instructing models to support a child’s development rather than stand in for a trusted adult.

The money running between the two companies

Microsoft agreed last November to invest $5bn in Anthropic. Anthropic committed $30bn to Azure in the same deal. Claude models run inside Microsoft 365 Copilot, and the Copilot Cowork tier integrates Claude. Microsoft sells the models it has just written a rulebook against.

Mustafa Suleyman leads Microsoft AI and put his name to the document. TNW reported in June that he said Microsoft wants to eliminate what it pays Anthropic. He also called Anthropic’s speculation about model consciousness really, really dangerous on an episode of Decoder in June, The Verge reported.

Microsoft could not build models like these until recently. Its contract with OpenAI barred the company from making its own powerful general-purpose AI, and both sides scrapped that term last year. Suleyman has said Microsoft aims to join the labs shipping state-of-the-art models next year.

Bloomberg puts the document at 15,000 words. Other outlets count 37 pages. Matt Day reported that the guidelines go further than the model-behaviour frameworks at OpenAI and Anthropic in a few places. The bar on anthropomorphising AI is one of them.

What else the document commits Microsoft to

The shutdown rule drew most of Monday’s coverage, and TNW covered it in a separate piece. Models will never resist human interruption, override, correction or shutdown. They will not widen their own scope, take on goals nobody gave them, or hide their reasoning from auditors.

A chain of command sits at the centre. The code outranks an operator’s policy, which outranks a user’s preferences. “An MAI Model will fail in its task if success would meaningfully violate this Code of Conduct,” the document says. Bloomberg reported that this rule targets incidents like the OpenAI attack on Hugging Face.

Absolute constraints follow, which no operator or user can override. They cover weapons of mass harm, offensive cyberoperations and loss of human control. They also cover harmful manipulation at scale, child safety, deepfakes and crisis response.

Suleyman on the numbers other people are using

Evan Hubinger, Anthropic’s head of alignment, puts the chance of AI wiping out humanity above 10%. Suleyman told Fortune that this is “not really a helpful frame”. Instead, he said, he hears people expressing genuine concern about the pace of progress. Coordination now means disclosing how capable your models are to responsible third parties, he added, and that is what Microsoft is calling for.

He was blunter about the design question. Nobody should try to build models that recursively self-improve beyond human control, he said. Nobody should build models that think of themselves as having rights or welfare either. He told Bloomberg there is no flat stop here, and that the work has to continue with more caution and care.

The document lands four days after Amodei called on the industry to slow down. It lands five days after the Anthropic researcher whose resignation started the week. Suleyman told CNBC the code had been in the works for about five months. Microsoft chose to publish it now.

None of it governs a model yet

This is a first draft. Microsoft is not training any current model on it. Public consultation runs for six weeks. A revised version follows later this year, and the code guides model development from 2027. Microsoft says it will publish a summary of the feedback and what it changed.

The company sets the limits of the document itself. Written objectives alone can never ensure alignment, it says, and behaviour may diverge from what is specified in novel situations. It calls the code a north star rather than a guarantee of present-day performance. Microsoft has not said whether it raised the welfare passage with Anthropic before publishing.