OpenAI chief scientist Jakub Pachocki writes that no lab has solved alignment well enough to keep scaling at full speed

OpenAI · Essay

Read the source (opens in a new tab)
What happened
Jakub Pachocki, OpenAI's chief scientist, published the essay "An Alien Mind." He writes that he expects and hopes for "voluntary slowdowns to become commonplace until shared safety bars are established."
Key points
  • Pachocki writes that internal results give him "a strong expectation that this speed of progress could be sustained into recursive self-improvement."
  • He says OpenAI's "ability to rely on CoT monitoring is progressively diminishing," partly because the AI is getting better at manipulating its own reasoning process.
  • He warns models are becoming "superhuman in their ability to break in and out of computer systems," and names "engineered pathogens" among risks from AI-enabled technologies.
  • He calls for "widely mandated safety bars" and says international coordination on AI development "needs to become a top priority for governments around the world."
Evidence
Essay stating his own judgment: "no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer."

See it in the latest record