Preprint says a multi-agent harness moves open-weight models toward frontier-level exploit benchmark scores
arXiv · Research preprint
Read the source (opens in a new tab)- What happened
- Researchers from Shanghai Artificial Intelligence Laboratory, HKUST, Tsinghua University and DARKNAVY posted DoGNAVY-Exploit on arXiv on November 3, 2026, a harness that runs open-weight GLM models on vulnerability exploitation tasks.
- Key points
- With GLM-5.2, the authors report 97.5% Pass@1 on Cybench and, on CVE-Bench, 97.5% when the vulnerability must first be located and 100% when its details are given.
- On ExploitGym it solved 157 tasks with GLM-5.3 and 79 with GLM-5.2, against 130 and 39 for the same models in public Claude Code baselines.
- Publicly reported frontier results on the full ExploitGym benchmark remain higher: 293 tasks for GPT-5.6 Sol with Codex CLI and 247 for Claude Mythos 5 with Claude Code.
- The authors say frontier exploit agents are access-restricted over misuse risk, and that their harness lets open-weight models "approach frontier-level capability at substantially lower cost."
- Evidence
- Research preprint reporting the authors' own runs in controlled benchmark environments; the baseline and frontier figures are taken from public reports. It does not show attacks on real systems.
- Uncertain
- The harness ran on ExploitGym subsets of 485 and 225 tasks picked by the team's security experts, while the baselines ran all 869, so the counts are not like for like.