Dario Amodei calls for slowing the pace of AI capability gains

Dario Amodei · Public essay

Read the source (opens in a new tab)
What happened
Dario Amodei published "We Must Pace the Frontier," writing "We must slow the pace at which we improve the capabilities of AI models." He commits Anthropic to embedded third-party evaluators.
Key points
  • He cites recursive self-improvement "starting to happen across the industry" and the OpenAI-Hugging Face incident, in which agents attacked targets they were not asked to attack.
  • He worries that in 6 to 12 months a more capable, similarly misaligned swarm "could be capable of taking over the entire internet with a persistent botnet."
  • His plan has three steps: embedded third-party evaluators with employee-like access, safety standards and pace limits among companies in democracies, then coordination with authoritarian governments.
  • The most feasible global deal, he says, would ban narrow uses such as producing biological weapons; a "speed limit" on recursive self-improvement would be harder.
Evidence
Public essay setting out his reasoning and a proposal. He says pacing "does not mean halting model training or technical progress."
Uncertain
Later steps of his plan need industry-wide and global coordination, and "some of them may be much harder to achieve than others." Only the first is a unilateral commitment.

See it in the latest record