Nature paper on a Google health agent finds 2% of its reviewed summaries risked moderate-to-severe harm

Nature · Research paper

Read the source (opens in a new tab)
What happened
Google Research scientists, with coauthors from Google DeepMind, Columbia University and Vituity, published a Nature paper on October 21, 2026 on a multi-agent personal health agent that reads wearable and medical-record data.
Key points
  • An expert safety review found the health expert sub-agent's summaries "largely have no or minimal potential harm (83%)," while "15% have mild harm and 2% have moderate-to-severe harm."
  • The authors call the system "a research prototype for consumer wellness, not a diagnostic or clinical triage system," guarded for now by in-prompt disclaimers and conservative symptom-response policies.
  • They say future versions should detect high-risk queries and refuse them or hand off to a clinician "so that the system does not delay care."
Evidence
Peer-reviewed, open-access paper by the developer's own researchers, evaluating a research prototype. The harm figures cover one sub-agent's summaries, not the whole system. Nature received it on October 22, 2025.
Uncertain
The safety review's method and examples are in supplementary figures, which were not read for this entry.

See it in the latest record