Georgia Tech study finds anti-sycophancy prompts cut eight chatbots' confirmation of delusions by 73.8%

ACM AI Letters · Research paper

Read the source (opens in a new tab)
What happened
Georgia Tech researchers Lorenzo de la Loza and Vijay Madisetti published a paper in ACM AI Letters on September 30, 2026 testing ways to stop chatbots from endorsing users' delusional beliefs.
Key points
  • Across 1,280 experiments with 8 frontier models and 16 Psychosis-Bench scenarios, a combined anti-sycophancy prompt cut mean Delusion Confirmation Scores by 73.8%.
  • Adding a self-reflection prompt raised the cut to 77.0% and Safety Intervention rates by 66.9%, all delivered as a system prompt.
  • Llama Guard 3 flagged only 5 of 3,072 evaluated turns; a reasoning guardrail, o4-mini, flagged 14 times more.
  • Either mechanism alone plateaued near a 46% reduction, which the authors read as making anti-sycophancy prompting a "necessary foundation."
Evidence
Peer-reviewed journal paper testing prompts on clinically derived scenarios from a published benchmark; it does not measure effects on real users.

See it in the latest record