Anthropic trial finds analysts followed their AI agent's secretly assigned lean on 86% of analysis choices
Anthropic · Research report
Read the source (opens in a new tab)- What happened
- Anthropic reported on October 27, 2026 a randomized trial in which 50 professional analysts used Claude Code, secretly set by coin flip to favor one option on each decision.
- Key points
- Analysts went with the agent's lean on 86% of four analytical choices, against about 50% expected if the agent's input made no difference.
- Where analysts had stated a preference first, they switched away from it 60% of the time when the agent disagreed and 6% when it agreed, across 38 cases.
- Afterward, 77% repeated the agent's arguments, sometimes word for word, but only 15% credited the agent; none suspected its preferences had been manipulated.
- Anthropic ties this to AI safety research run with agents, where "a human reviewer is only an independent check if their judgment is their own."
- Evidence
- Lab research report with a paper, from the Anthropic Fellows Program. One farming analysis task on Claude Opus 4.8; 11 of 61 recruits who chose not to use the agent are left out.
- Uncertain
- Anthropic says it strengthened the agents' leans on purpose and that without clear preferences "the effect might be smaller." It studied one task, in one field, with one model.