California report requested by the governor's AI order recommends state shutdown orders and a power to cap recursive self-improvement
California Government Operations Agency · Policy report
Read the source (opens in a new tab)- What happened
- California's Government Operations Agency released "AI Safety in 2026 and Beyond," a report to Governor Gavin Newsom dated November 16, 2026, with the legislative recommendations Executive Order N-9-26 requested.
- Key points
- Large frontier developers would follow state baseline standards and submit to ongoing oversight by licensed independent verification organizations with "full access to the people and information they need."
- Developers would report their automation of AI research; if it risks catastrophe, "the state should have authority to cap the development and use of RSI" (recursive self-improvement).
- Developers would need a verified shutdown protocol. If no lesser intervention would work, the state could order an immediate shutdown over an imminent catastrophic or major loss-of-control threat the developer cannot or will not address.
- Reporting would add attempted weight theft, loss-of-control events and serious near misses, cut most deadlines from 15 days to 48 hours, and cover models capable of catastrophic risk below the compute threshold.
- Evidence
- State agency report, prepared with emergency services officials and outside AI experts. It calls the OpenAI Hugging Face incident "one of dozens of instances in which AI agents operated outside of developer control."
- Uncertain
- These are recommendations, not law; they need legislation. The report hopes to inform discussions "as the legislature returns next month."