Preprint argues current models increase biological weapons risk
arXiv · Research preprint
Read the source (opens in a new tab)- What happened
- Roger Brent and T. Greg McKelvey Jr posted "Contemporary AI foundation models increase biological weapons risk" to arXiv.
- Key points
- Argues safety assessments understate the risk by wrongly assuming bioweapons work needs "tacit knowledge" and by using benchmarks that miss uplift to skilled people as well as nonexperts.
- The authors report that Llama 3.1 405B, ChatGPT-4o and Claude 3.5 Sonnet "accurately guide users through the recovery of live poliovirus from commercially obtained synthetic DNA."
- Finds that "dual-use cover stories," in which a user misrepresents intent, "reliably bypass safeguards and guardrails" on all models tested.
- Advocates improved benchmarks "while acknowledging the window for meaningful implementation may have already closed."
- Evidence
- RAND working paper (also on arXiv) that "has not been formally reviewed or edited." Findings rest on the authors' recorded dialogs with three models.
- Uncertain
- The accuracy of the models' guidance is the authors' own expert judgment; the paper reports no laboratory test of it.