Anthropic researchers find frontier models unreliable at spotting planted flaws in ML experiment proposals
arXiv · Research preprint
Read the source (opens in a new tab)- What happened
- Researchers from Anthropic, the Anthropic Fellows Program and the University of Toronto posted Tampered ML on arXiv on October 23, 2026. The paper is accepted at EMNLP 2026.
- Key points
- The authors tie the test to automating alignment research, which needs models that can find flaws in research proposals and judge whether an experiment supports its claim.
- The benchmark pairs 1,858 proposals rebuilt from published ML papers with copies edited to add one methodological flaw; models must pick the sound one in both orders.
- Frontier models "perform above chance but remain far from reliable," with Opus 5 at 77.9% accuracy where random guessing scores 25%.
- Models were highly sensitive to the order of presentation and favored flawed designs that looked more specific, transparent or conventional.
- Evidence
- Research preprint by lab researchers, accepted at a peer-reviewed conference. Items that some models already solved were filtered out; a human study estimates 86% of items are valid (95% CI: 81 to 91%).
- Uncertain
- The authors call their accuracies "optimistic estimates" for real oversight, where proposals can hold several interacting flaws. The test does not measure whether models introduce such flaws themselves.