Study finds AI reasoning models make over 34% more false-positive diagnoses than matched standard models
Nature Communications · Research paper
Read the source (opens in a new tab)- What happened
- Researchers from the University of Hong Kong, Tsinghua University, Harvard Medical School and others published a Nature Communications study on October 29, 2026 comparing reasoning and standard AI models in clinical diagnosis.
- Key points
- On a benchmark built from real-world electronic health records, six matched model pairs across 12 specialties were compared.
- Reasoning models made more accurate primary diagnoses but over 34% more false positives and about 40% lower reproducibility; the authors say the trade-off held across prompt designs and settings.
- In a 15-physician study, letting the reasoning model widen the diagnosis while the clinician checked it raised accuracy by 20% and cut false positives by 44%.
- The authors say pairing reasoning models with physician oversight needs "prospective validation before clinical deployment."
- Evidence
- Peer-reviewed, open-access paper. The figures are from its abstract; the full results and the specific models tested were not reviewed for this entry.
- Uncertain
- The authors say the results were "obtained under controlled retrospective conditions," not in live patient care.