Anthropic researchers find frontier models unreliable at spotting planted flaws in ML experiment proposals

arXiv · Research preprint

Read the source (opens in a new tab)
What happened
Researchers from Anthropic, the Anthropic Fellows Program and the University of Toronto posted Tampered ML on arXiv on October 23, 2026. The paper is accepted at EMNLP 2026.
Key points
  • The authors tie the test to automating alignment research, which needs models that can find flaws in research proposals and judge whether an experiment supports its claim.
  • The benchmark pairs 1,858 proposals rebuilt from published ML papers with copies edited to add one methodological flaw; models must pick the sound one in both orders.
  • Frontier models "perform above chance but remain far from reliable," with Opus 5 at 77.9% accuracy where random guessing scores 25%.
  • Models were highly sensitive to the order of presentation and favored flawed designs that looked more specific, transparent or conventional.
Evidence
Research preprint by lab researchers, accepted at a peer-reviewed conference. Items that some models already solved were filtered out; a human study estimates 86% of items are valid (95% CI: 81 to 91%).
Uncertain
The authors call their accuracies "optimistic estimates" for real oversight, where proposals can hold several interacting flaws. The test does not measure whether models introduce such flaws themselves.

See it in the latest record