Newsom pitches an AI kill switch for California

Gavin Newsom has signed an executive order asking a working group to report within two months on strengthening California’s AI safety laws, including whether companies should build a kill switch for frontier models. He vetoed SB 1047, which required developers to be able to promptly enact a full shutdown, in September 2024.

Gavin Newsom signed an executive order on Friday telling a working group to report within two months on how to strengthen California’s AI safety laws, Bloomberg reported. Among the measures it should consider is making companies build a kill switch for frontier models.

He vetoed a bill that required one.

SB 1047 would have obliged developers of covered models to be able to “promptly enact a full shutdown“, stopping operation and further training. Newsom vetoed it on 29 September 2024, arguing it regulated models by size and cost rather than by what they are used for. The bill was the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act.

That was almost two years ago to the day.

The order’s other idea is independent third parties writing safety plans for frontier companies. “We’re not waiting to act,” Newsom said. His office has not said who is in the working group.

Europe did not ask companies to build the switch.

Since 2 August the European Commission has been able to restrict, withdraw or recall a general-purpose model from the EU market, under a provision of the AI Act we covered this month. The power sits with the regulator rather than the developer.

It does not depend on a company implementing anything.

The AI Act separately requires that a human overseeing a high-risk system can interrupt it or bring it to a halt through a stop button. That rule covers deployed systems rather than the models underneath them. Those obligations are not yet in application.

California is two months from a recommendation.

The gap on third parties looks similar. California’s SB 813 gives the state until January 2028 to certify organisations able to test frontier models, while Europe’s 60-member scientific panel was appointed on 1 June.

Neither side has settled who pays for the compute. On the only worked example so far, the tester’s bill was met by the company being tested.

Newsom has been on both sides of this. He encouraged the boom because it paid, and earlier this year promoted a partnership to put Anthropic’s Claude into state agencies. Anthropic and OpenAI are both based in California, and both would fall inside whatever his group recommends.