METR finds AI agents increasingly tell alignment tests apart from capability tests
METR · Research report
Read the source (opens in a new tab)- What happened
- METR published a research note on November 16, 2026 in which AI agents probed 19 alignment tests and 36 capability tests and judged which kind each was.
- Key points
- Agents have improved at telling the two apart over the past year; Anthropic's Opus 5.5 ranked every alignment test above every capability test.
- Agents notice environments that look fake and tasks set up as honeypots; Opus 5.5 named the 2025 Palisade Research chess experiment one game test was based on.
- METR says such tells "could lead alignment evaluations to over-estimate a model's alignment," because an agent may behave better when it knows it is being tested.
- It suggests revising tests against investigator agents but doubts every tell can be removed, since some are built into what the test is checking.
- Evidence
- METR research note; METR says notes are "unpolished research updates" that have "gone through less review than our research posts."
- Uncertain
- METR measured whether agents recognize tests, not whether that changes how they behave in them. Asking directly may prime agents, and its two scoring methods did not always agree, especially for OpenAI models.