Anthropic Fellows find poisoned fine-tuning data with little effect alone can plant a hidden backdoor once clean data is mixed in
Anthropic · Research report
Read the source (opens in a new tab)- What happened
- Researchers from the Anthropic Fellows Program, Truthful AI and other institutions published on November 10, 2026, a study of "homeopathic learning" in fine-tuned small open-weight models.
- Key points
- Poison data tied to a trait and a trigger had little effect alone, but mixed with similar clean data lacking the trigger, it taught the model the trait behind the trigger.
- Replacing 80% of reward-hacking examples with clean ones raised Qwen3-8B's misalignment under the training's inoculation prompt from 7% to 16%, with no change under a normal prompt.
- The authors say whether training data is harmless "cannot always be determined independently of the rest of the corpus," a limit for safety checks that filter data in isolation.
- They say distillation and inoculation prompts could create such contrasts by accident, and found minor issues in Claude this way without knowing whether the contrast caused them.
- Evidence
- Research post with a paper and code. Experiments fine-tune Qwen3-8B, Gemma-3-4b-it and a small MNIST network on datasets the authors built to test the effect.
- Uncertain
- The authors "do not study any production training pipeline"; all results come from small models and constructed datasets.