Goodfire deploys cyber monitors on Kimi K3 and GLM 5.3 that it says beat an LLM judge at 50x lower cost
Goodfire · Research report
Read the source (opens in a new tab)- What happened
- Goodfire reported on October 8, 2026 that it built probe-based cybersecurity monitors for the open-weight models Kimi K3 and GLM 5.3 and deployed them on a production inference stack.
- Key points
- A probe reads the model's internal activations as a cheap first-line filter and sends only suspicious exchanges to an LLM judge, matching the judge at "93% recall at 5.5% FPR."
- Goodfire says the cascade costs about 50x less, "at under $200 per million exchanges," and adds no latency, so risky actions can be blocked before a tool is called.
- In FAR.AI's red-teaming of the deployed system, 140 jailbreak strategies produced 66 universal jailbreaks without the monitor and none with it; jailbroken interactions fell from 700 to 18.
- Citing "recent incidents involving agentic coding models," Goodfire argues open models "don't come with a full safety stack" and monitoring like this should be a default for serving them.
- Evidence
- Company research report on Goodfire's own evaluations of a system it sells, plus an external red team by FAR.AI. The recipe carried over to GLM 5.3 using only Kimi K3 data.
- Uncertain
- FAR.AI calls its testing "preliminary": two days, a static non-adaptive set of jailbreaks, chat contexts only. Across 1,120 non-universal interactions, 0.1% to 1.6% still got through.