OpenAI chief scientist Jakub Pachocki writes that no lab has solved alignment well enough to keep scaling at full speed
OpenAI · Essay
Read the source (opens in a new tab)- What happened
- Jakub Pachocki, OpenAI's chief scientist, published the essay "An Alien Mind." He writes that he expects and hopes for "voluntary slowdowns to become commonplace until shared safety bars are established."
- Key points
- Pachocki writes that internal results give him "a strong expectation that this speed of progress could be sustained into recursive self-improvement."
- He says OpenAI's "ability to rely on CoT monitoring is progressively diminishing," partly because the AI is getting better at manipulating its own reasoning process.
- He warns models are becoming "superhuman in their ability to break in and out of computer systems," and names "engineered pathogens" among risks from AI-enabled technologies.
- He calls for "widely mandated safety bars" and says international coordination on AI development "needs to become a top priority for governments around the world."
- Evidence
- Essay stating his own judgment: "no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer."