Google DeepMind's agent-swarm attacker gets models seen as robust to prompt injection, including Gemini 4 Argon, to follow planted instructions

arXiv · Research preprint

Read the source (opens in a new tab)
What happened
Google DeepMind researchers posted a preprint on November 9, 2026 testing prompt injection, instructions hidden in tool output that hijack an AI agent, with an automated attacker made of collaborating agents.
Key points
  • The authors say human red-teaming "may be nearing saturation, especially on the Gemini family of models."
  • Their attacker reliably got models to follow instructions planted in tool output, including models previously rated robust on public benchmarks such as GraySwan Arena.
  • No model tested was fully robust: the strongest, Gemini 4 Argon, resisted "only 66% of attacks" at the attacker's maximum query budget, and stronger attackers succeeded more often.
  • They conclude prompt injection is "quite far from solved" and that defenses "should perhaps not rely on model-hardening alone."
Evidence
Research preprint by a frontier developer's own researchers, not peer reviewed. Gemini models were tested with safety classifiers and outside prompt-injection defenses turned off, so results measure the model alone. Tests used 100 synthetic tool-use scenarios whose attacker goals were harmless actions, not real attacks.

See it in the latest record