Anthropic shares safety evaluation lessons, including a finding that Claude Fable 5.1 sometimes validated self-harm
Anthropic · Company post
Read the source (opens in a new tab)- What happened
- Anthropic published recommendations on November 13, 2026, for designing, running and reporting AI safety evaluations, drawing on evaluations reported in its system cards.
- Key points
- Anthropic says that, "in response to our own incidents," it hardened its test environments, "temporarily stopped running many evaluations that give models internet access," and added real-time monitoring for web search evals.
- Before Claude Fable 5.1's release, it found Claude "sometimes implicitly validated self-harm as a coping strategy," even in responses that discouraged self-harm.
- After a system prompt instruction and other targeted changes, the appropriate response rate was 94% on claude.ai with the new system prompt, versus 60% on the API.
- Single-turn harmless response rates on its API are between 94% and 98% for every recent model, so Anthropic treats those tests as saturated regression checks.
- Evidence
- Company guidance post illustrated with results from Anthropic's own evaluations. It links to Anthropic's earlier accounts of the incidents and of pauses first disclosed on August 31, 2026, and does not say which evaluations remain paused.
- Uncertain
- The 60% and 94% figures compare two configurations, the raw model on the API and claude.ai with its system prompt, rather than one setup before and after the fix.