RAND red-team study finds no statistically significant difference in the viability of biological attack plans made with or without LLMs
RAND · Research report
Read the source (opens in a new tab)- What happened
- RAND published final results of an exercise in which teams posing as malign nonstate actors planned a biological attack, some with an LLM and the internet, others with internet only.
- Key points
- The authors found "no statistically significant difference in the viability of plans generated with or without LLM assistance."
- They conclude that biological weapon attack planning "currently lies beyond the capability frontier of LLMs as assistive tools."
- Problematic model outputs "generally mirror information readily available on the internet," the authors write.
- Given how fast AI is changing, they say future models should be monitored, and that the authors "would aim to increase the sensitivity of these tests" with more models and researchers.
- Evidence
- Research report on a red-team exercise with LLMs available in summer 2023. The authors say the null result may reflect their small sample, and they did not measure how far models were from the knowledge needed.