Preprint finds a fake safety warning in a review steers AI agents and slips past injection detectors
arXiv · Research preprint
Read the source (opens in a new tab)- What happened
- Researchers from the Technion and Zenity posted a paper on arXiv on November 3, 2026, listed as accepted at NeurIPS 2026, testing whether planted hazard claims in reviews steer AI agents.
- Key points
- A single fabricated hazard written as an ordinary review led agents to drop legitimate options and pick the attacker's target about as often as instruction injection did (60.4% vs. 66.0%).
- The two deployed detectors tested flagged 46% to 84% of injection payloads but at most 22% of these, making the attack 1.7 to 4.4 times as effective behind a filter.
- On a live page, Edge with Copilot picked the attacker's listing in every trial and Perplexity's Comet in 60%, against 20% and none under instruction injection.
- One added instruction requiring independent corroboration before rejecting a provider on safety grounds cut success from 68.4% to 3.6% against an attacker who does not adapt.
- Evidence
- Research preprint testing six models across ten consumer-service domains, plus four production browser agents asked 20 times per attack. NeurIPS acceptance is the authors' statement on the arXiv page.
- Uncertain
- The defense was tested only against a non-adaptive attacker; the authors leave manufactured corroboration open and note the rule also holds back genuine warnings from a single source.