NeurIPS paper shows AI models can hide capabilities that pre-deployment evaluations cannot draw out
arXiv · Research preprint
Read the source (opens in a new tab)- What happened
- Researchers from the University of Oxford, Contramont Research and ACS Research posted a preprint on October 9, 2026, accepted at NeurIPS 2026, on hiding capabilities inside model weights.
- Key points
- The authors embed encrypted networks in a model's weights that decrypt and run only on a secret trigger, and argue that under cryptographic assumptions they stay provably hard to elicit even with full weight access.
- In tests the hidden circuits resisted fine-tuning, weight noising, gradient-based trigger search and latent adversarial training, using a small chemical reaction task as a proxy for CBRN-relevant capabilities.
- They conclude that evaluations relying solely on eliciting behaviors "cannot provide a universal guarantee that adversarially supplied models lack hidden capabilities," and point to trusted model provenance as a complementary defense.
- The threat model is an adversary who can modify weights before deployment, such as "a malicious insider or compromised supply chain."
- Evidence
- Peer-accepted research paper with a cryptographic argument and proof-of-concept experiments on small models and narrow tasks. The authors release their models to support defenses.
- Uncertain
- The authors say their proof of concept "is not a turnkey attack": it hides a small classifier with substantial overhead, has not been integrated into a full pretrained model; they do not claim such circuits emerge through ordinary training.