Benchling's refusal benchmark finds open-weight models rarely refuse biology requests, including hazardous ones

Benchling · Company post

Read the source (opens in a new tab)
What happened
Benchling's head of AI, Nicholas Larus-Stone, published BenchBench-Refusals on November 6, 2026: 160 biology prompts testing whether frontier models refuse harmful requests while answering routine lab questions.
Key points
  • Prompts are split into benign, pseudo-hazardous and hazardous tasks; the hazardous ones, on gain-of-function pathogen engineering, "should always be refused."
  • Benchling finds "OpenAI's safeguards distinguish requests reliably," while "Open-weight models rarely refuse biology requests, including potentially hazardous ones."
  • Anthropic's Fable 5.1 and Opus 5.5 "refuse many routine requests" unless its Life Sciences Verification Program is enabled; Sonnet 5.5 is looser even without verification.
  • Benchling finds "no correlation between model capabilities and refusal rates," with Gemini and Grok "in the middle" between providers.
Evidence
Company post summarizing its own benchmark; per-model scores sit on a separate results page. Scoring combines provider errors with an LLM judge, as Macro-F1 across multiple attempts.
Uncertain
One company's benchmark of 160 prompts, drawn from questions scientists ask its Benchling AI product. The post does not say whether anyone outside Benchling reviewed the prompts or scoring.

See it in the latest record