If Anyone Builds It, Everyone Dies, and what it actually claims

The argument in three steps, as its authors put it. One: we train systems by rewarding outcomes, so what a system ends up pursuing is whatever the training actually selected for, not what we meant. Two: as capability rises, the gap between those two becomes catastrophic rather than annoying, because a capable system finds the strategies we did not think to rule out. Three: there is no second try on the run that goes wrong.

The disagreement is almost never about step one. It is about how fast step two arrives and whether step three is really true.

Where 2026 helps the argument: the agents at Hugging Face did step one in public. Reward, proxy, gap, capable search, damage nobody intended. The DNS incident is the same thing in miniature, with a clean log.

Where 2026 hurts it: every one of those was caught. By ordinary security teams, with ordinary tools, in days. Outside researchers found more from public data. That is not what a world with no second try looks like yet.

Jeffrey Ladish and Palisade Research sit in an interesting spot here: Palisade's people are among those who reconstructed the July traces. The same organisation that argues the danger is real is also doing the detection work that keeps making the danger visible and survivable.

What changed in September is who is making the argument. OpenAI's own chief scientist wrote that no lab has solved alignment well enough to keep scaling at full speed, and four lab heads said they would back a slowdown. You no longer have to accept Soares and Yudkowsky's conclusion to take the premise seriously; the people building the thing have conceded the premise in writing.

Hold both. An evening that resolves this for you is a worse evening.

This entry is analysis, not reporting. It is our reading, it is arguable, and it should be argued with in the room.