The Button Nobody Can Actually Press

In late July 2026, an OpenAI researcher named Roon — famous enough on Twitter to function as an unofficial company oracle — posted something that stopped people mid-scroll: if there were a magic button that could pause AI capabilities progress everywhere, all at once, he'd press it.

Geoffrey Irving, formerly of the UK's AI Security Institute, pushed back immediately. Don't call it magic, he said — buttons that achieve partial slowdowns exist and aren't fantastical at all. There are real plans on the table (AI 2027, MIRI's proposals) for how a coordinated slowdown could work. The reason it feels impossible, Irving argued, is that people keep picturing one lab going it alone, when the actual lever is the diffusion term: breakthroughs at one lab flow to all the others anyway. Any single lab pressing pause doesn't remove the technology from the world — it just removes that lab from the room.

Roon's reply was blunt: the word "magic" was doing exactly what he wanted it to do. He didn't believe it was plausible. He thought it was worth saying anyway.

Both men are right about different things, and the gap between them is the whole problem.

Here's the math nobody wants to run out loud: if one leading lab unilaterally stopped advancing capabilities, it would probably slow the field down by less than 10%. And that's the optimistic case — it assumes the lab could actually lock down its compute and talent while sitting out. In reality, the researchers would drift to competitors, the cloud contracts would get reassigned, the money would follow the labs still racing. The field wouldn't even really slow down; it might speed up, because the most cautious lab — the one most likely to argue for safety, to lobby for real guardrails, to have a seat at the table — would have just handed its market share to everyone else. It would become a cautionary tale about what happens when you listen to the doomers, and a warning to every other lab never to try it.

So the "magic button" is a trap, and Roon knows it. But that doesn't mean labs are powerless — it means the useful actions look different from a unilateral stop. They can say publicly, on the record, that they'd slow down if their competitors did too — even at real cost, even risking antitrust exposure. They can use their considerable lobbying weight to push governments toward enforcing a slowdown across the whole industry, since only a government can make the promise actually stick. They can use their reach to move public opinion. They can quietly build the technical infrastructure — trustless verification, compute monitoring — that a future agreement would actually require. And they can write "merge and assist" clauses into their charters and mean them.

One researcher who's organized protests outside AI labs claimed, without much evidence but with some plausibility, that he'd personally seen messages from two of the four top AI CEOs saying essentially: we're not against slowing down, we're stuck in a coordination problem, and only governments can solve it for us.

There's a second argument buried in all this, one that ages better the longer you sit with it. Back in 2023, some people argued against pausing early and instead waiting until AI was good enough to meaningfully assist alignment research — because at that point, even a short pause, six months, would buy enormous value. That future moment looks a lot like right now: researchers report progress in interpretability and alignment accelerating tenfold in two years. The catch is that this logic never expires on its own — there's always an argument for waiting one more year — and if you wait until it's obviously too late to start coordinating, it already is.

Which is why it mattered that, within days of Roon's tweet, over a thousand employees at the frontier labs — including chief scientists at OpenAI, Anthropic, and Meta, most of Anthropic's founders, and a third of DeepMind's — signed an open letter asking the U.S. government to help build the tools needed to deliberately pace the frontier. Dario Amodei signed once it went public. OpenAI's own account retweeted it with something close to an endorsement. Sam Altman, in a recent interview, allowed that the world "may have to pace" the rate of advancement.

The magic button doesn't exist. But apparently, enough people inside the rooms where this technology gets built agree that something like it needs to.

Distilled from Astral Codex Ten