SecureBio and UK AI Security Institute find cheap fine-tuning undoes viral-protein filtering in protein language models

arXiv · Research preprint

Read the source (opens in a new tab)
What happened
Researchers from SecureBio, the UK AI Security Institute, the Broad Institute, and Oxford posted a preprint testing whether filtering viral proteins from the training data of ESM-2 protein language models is an effective and durable safeguard.
Key points
  • Filtering "meaningfully degrades" modeling of viral proteins while "largely preserving performance on non-viral proteins," with the effect "concentrated among proteins from human-infecting viruses" and growing as models scale from 8M to 3B parameters.
  • "Fine-tuning on sequences from the removed viral families restores the lost performance," costing "about 80 USD of H100 time" even for the largest model.
  • They conclude the method is "moderately effective at selectively degrading performance on viral proteins, but not durable against fine-tuning with modest computational resources and access to relevant data."
Evidence
An arXiv preprint, not yet peer-reviewed, using the open ESM-2 model family.

See it in the latest record