Benchling's refusal benchmark finds open-weight models rarely refuse biology requests, including hazardous ones
Benchling · Company post
Read the source (opens in a new tab)- What happened
- Benchling's head of AI, Nicholas Larus-Stone, published BenchBench-Refusals on November 6, 2026: 160 biology prompts testing whether frontier models refuse harmful requests while answering routine lab questions.
- Key points
- Prompts are split into benign, pseudo-hazardous and hazardous tasks; the hazardous ones, on gain-of-function pathogen engineering, "should always be refused."
- Benchling finds "OpenAI's safeguards distinguish requests reliably," while "Open-weight models rarely refuse biology requests, including potentially hazardous ones."
- Anthropic's Fable 5.1 and Opus 5.5 "refuse many routine requests" unless its Life Sciences Verification Program is enabled; Sonnet 5.5 is looser even without verification.
- Benchling finds "no correlation between model capabilities and refusal rates," with Gemini and Grok "in the middle" between providers.
- Evidence
- Company post summarizing its own benchmark; per-model scores sit on a separate results page. Scoring combines provider errors with an LLM judge, as Macro-F1 across multiple attempts.
- Uncertain
- One company's benchmark of 160 prompts, drawn from questions scientists ask its Benchling AI product. The post does not say whether anyone outside Benchling reviewed the prompts or scoring.