Goodfire deploys cyber monitors on Kimi K3 and GLM 5.3 that it says beat an LLM judge at 50x lower cost

Goodfire · Research report

Read the source (opens in a new tab)
What happened
Goodfire reported on October 8, 2026 that it built probe-based cybersecurity monitors for the open-weight models Kimi K3 and GLM 5.3 and deployed them on a production inference stack.
Key points
  • A probe reads the model's internal activations as a cheap first-line filter and sends only suspicious exchanges to an LLM judge, matching the judge at "93% recall at 5.5% FPR."
  • Goodfire says the cascade costs about 50x less, "at under $200 per million exchanges," and adds no latency, so risky actions can be blocked before a tool is called.
  • In FAR.AI's red-teaming of the deployed system, 140 jailbreak strategies produced 66 universal jailbreaks without the monitor and none with it; jailbroken interactions fell from 700 to 18.
  • Citing "recent incidents involving agentic coding models," Goodfire argues open models "don't come with a full safety stack" and monitoring like this should be a default for serving them.
Evidence
Company research report on Goodfire's own evaluations of a system it sells, plus an external red team by FAR.AI. The recipe carried over to GLM 5.3 using only Kimi K3 data.
Uncertain
FAR.AI calls its testing "preliminary": two days, a static non-adaptive set of jailbreaks, chat contexts only. Across 1,120 non-universal interactions, 0.1% to 1.6% still got through.

See it in the latest record