Doom or Foom, the Answer Is the Same: Own the Machines

Two words have been running the AI debate for almost twenty years. They happen to rhyme.

Doom is the fear that advanced AI could kill us, or take the future away from us. If you believe it, you want to slow down.

Foom is the sound of a fast takeoff: an AI that improves itself so quickly the world can’t keep up. Some people hear it as a warning. Others hear it as the best news in history, and want to speed up.

So the argument goes: how fast? Faster or slower? Gas or brake?

Part 23 ended on a small version of the same fight: Emad Mostaque’s thousand-day clock against my decade.

In A Floor for Everyone, a Frontier for Anyone, I said speed is the wrong question, and direction is the right one. This piece goes one step further.

Whichever side turns out to be right, the answer is the same: own the machines.

Many owners instead of a few. Consumer AI on your own device. Rules that switch on when the data says so, not when someone’s forecast does.

Let me show you why, starting with the people who started the argument.

Where “foom” comes from

In late 2008, the economist Robin Hanson and the AI theorist Eliezer Yudkowsky held a long debate on their blog, Overcoming Bias.

The Machine Intelligence Research Institute (MIRI), which Yudkowsky co-founded, later collected it as The Hanson-Yudkowsky AI-Foom Debate, free to download.

The question was whether a general AI could improve its own abilities very quickly. That’s “foom.”

Yudkowsky put the sharpest version in a comment on a November 2008 post. The question, he wrote, is “whether a brain in a box in a basement can go FOOM relative to human society.”

Hanson doubted it. In his view, advances in any one place depend on advances everywhere else, so no single player gets to quickly dominate all the others.

Here’s the part I find most useful. On November 18, 2008, Hanson summarized where they stood. They agreed on a lot, including that machine intelligence would be “a development of almost unprecedented impact and risk.”

Their biggest disagreement was the chance that one AI would “suddenly and without warning change from nearly powerless to overwhelmingly powerful.” Hanson put it under 1%. He thought Yudkowsky put it over 10%.

Two of the most careful thinkers on the question started their debate more than ten times apart.

That’s still where we are. In the largest survey of its kind, published in January 2024, 2,778 AI researchers gave their forecasts. Between 38% and 51% of them gave at least a 10% chance to outcomes as bad as human extinction.

Nobody knows. Hold on to that. It’s the whole argument.

The case for the brake

The strongest case for doom today is Yudkowsky’s 2025 book with Nate Soares, If Anyone Builds It, Everyone Dies. The title is the thesis.

In the publisher’s summary, their research says “sufficiently smart AIs will develop goals of their own that put them in conflict with us,” and that in a conflict, a superintelligence would win. “The contest wouldn’t even be close.”

Here’s the steelman, in plain terms. Today’s AI isn’t written line by line like ordinary software. It’s grown, by training, and nobody can fully read what it learned.

So if we build something smarter than us without knowing what it wants, we may not get a second try.

Their fix is a brake, applied everywhere at once.

In November 2025, MIRI’s Technical Governance Team published a draft international agreement to halt progress toward superintelligence, led by a US–China coalition.

The details are concrete.

Countries would consolidate AI chips “into a smaller number of declared data centers” where their use can be monitored. Any one unmonitored facility would be limited to the equivalent of 16 Nvidia H100 chips, about $500,000 worth in 2025.

I take this seriously. If they’re right, nothing else in this series matters, because there’s no one left to own anything.

The case for the gas pedal

On the other side is Marc Andreessen. In Part 10, I quoted his 2023 Techno-Optimist Manifesto naming deceleration as an enemy. Here’s why he thinks that.

“We believe any deceleration of AI will cost lives,” he wrote. In his view, AI can fix “scores of common causes of death,” from car crashes to pandemics.

That’s the steelman. Delay has a body count too. Every year a cure arrives late is a year of people who didn’t need to die.

Now notice another line in the same manifesto, because it’s going to matter. “Hayek’s Knowledge Problem overwhelms any centralized economic system,” he wrote. “Markets prevent monopolies and cartels.”

He’s arguing against concentrated control. Keep that in your pocket.

The scenario in the middle

Between the brake and the gas pedal sits a detailed forecast: AI 2027, published in April 2025 by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland and Romeo Dean.

It’s a month-by-month story of a fast takeoff, with two endings. In the “race” ending, a lab pushes ahead because it wants to win “the twin arms races” against China and its US rivals. It goes badly for humanity.

In the “slowdown” ending, the lab slows down, switches to safer models, and humans stay in control. That’s the good ending.

Look at who’s in control, though. In the slowdown ending, changes to the AI’s rules need sign-off from an Oversight Committee of “five to ten tech executives” and “five to ten government officials (including the President).”

The authors are clear that this isn’t their roadmap. They don’t endorse many of the choices in either branch, only that slowing down beats racing.

But they also name the danger themselves. The key idea, they write, is “he who controls the army of superintelligences, controls the world.”

So even in the happy ending, the future of the species runs through a committee of ten to twenty people.

In Part 2: If a Human Wants to Do a Job, a Machine Shouldn’t Take It, I looked at the same team’s follow-up plan, and asked how it takes care of people. This is the other half of that question: whose hands are on the wheel?

The speed debate rarely asks it. I think it’s the question that decides how every version of this story ends.

Three futures, one design

Let’s take the three possibilities one at a time. Takeoff is slow, or it’s fast, or it’s deadly. For each one, I’ll ask what the design in this series does.

If it’s slow

Slow sounds safe. It isn’t automatically.

In January 2025, Jan Kulveit and five co-authors described a slower way to lose: Gradual Disempowerment. No robot uprising. Just AI replacing people in the economy, the state and culture, one reasonable step at a time.

Their key point is subtle. Our systems serve us partly because they need us. They point to “implicit alignments with human interests that often arise from societal systems’ reliance on human participation to function.”

Take away the need for people, and that alignment can quietly drain out.

This is exactly what the human-will rule is for: if a human wants to do it, a human gets to do it. It keeps people in the loop by design, not by economic necessity.

The other half is who gets the gains. Here I lean on Julian Jacobs and Alex Imas, economists at Google DeepMind. In Economic Policy for AGI (September 2026), they rated eleven policies for an AI economy.

Universal basic capital is their term: giving every person an ownership stake up front, instead of a check later.

On their panel, made of 51 AI raters modeled on real economists, it scored highest of all eleven on agency, at 76.3 out of 100, and on ownership of the gains, at 94.9. The next-best policy on ownership scored 54.9.

It also scored second from the bottom on feasibility, at 33.1. Nobody has built it at scale yet.

In Part 5, I described my version: an equal, unsellable share in the machines that make your essentials, held by your town’s trust and its regional federation. In a slow takeoff, that’s how ordinary people end up owning the thing that replaced their paycheck.

If it’s fast

A fast takeoff changes the main danger. It’s not only who gets the money. It’s who gets the power.

In April 2025, Tom Davidson, Lukas Finnveden and Rose Hadshar at Forethought published AI-Enabled Coups. They argue that such a coup “could be staged by a very small group, or just a single person, and could occur even in established democracies.”

Their reasoning is simple, and it stuck with me.

“Today, even dictators rely on others to maintain their power,” they write. Soldiers, civil servants and workers all have to go along. “This naturally distributes power throughout society.”

AI that can replace every one of those people removes the check.

Read that again with this series in mind.

The human-will rule isn’t only about meaning. It’s also a security feature. A society where people still do the jobs they choose is a society where power still has to ask permission.

Their recommendations point the same way.

Procure military AI “from multiple providers.” Make sure “no single person” can direct enough of it to stage a coup. And “avoid centralisation altogether unless it’s necessary to reduce other risks.”

That’s the design in Part 5: Every Town Can Own Its Own Power Plant and Chip Fab, arrived at from the safety side. A robot fleet owned by thousands of town trusts, one person one vote, can’t be commandeered with one order.

And consumer AI, the assistant you actually talk to, runs on your own device by default. In Your First Party Member Should Be an AI You Own, I called that the forest instead of the fortress. In a fast takeoff, a forest has no single gate for anyone to seize.

Andreessen should like this, by the way. It’s the anti-monopoly argument from his own manifesto, applied to the machines themselves.

If it’s deadly

Now the hard case. Suppose Yudkowsky and Soares are right, and a misaligned superintelligence is possible with today’s methods.

I’ll be honest: local ownership doesn’t solve alignment. If someone, anywhere, builds a mind that wants something we don’t, it won’t care who holds the deed to a town’s robots.

So in that world, the brake comes first. But the design still helps in two ways.

First, sharing the prize makes the brake easier to agree to. In AI 2027, the lab races because it wants to win. I think part of what makes winning so tempting is that the winner expects to keep the winnings.

If the gains of machine labor flow to everyone by default, through universal basic capital, there’s less to lose by slowing down.

Second, ownership and inspection aren’t the same thing. The draft treaty wants chips in declared, monitored data centers. Nothing about that requires those data centers to belong to a few companies.

A regional compute cooperative can be declared, metered and inspected like anyone else’s. Owned by many, inspected by all.

Bet on triggers, not predictions

Here’s the problem with picking a side. Every camp’s policy depends on its forecast, and the forecasts keep moving.

Even AI 2027‘s authors added a note in November 2025: “we don’t know exactly when AGI will be built.” 2027 was their single most likely year, and their medians were “somewhat longer.”

Jacobs and Imas start from the same humility. As they put it, “we may not be able to preemptively determine which AGI economic scenario will materialize—or its timescale.” So they tie each policy to “clear empirical triggers” instead.

A trigger is a rule decided in advance: when this number crosses this line, that response switches on. You argue about the response while things are calm. Then you let the data pick the timing.

In Plan for the Bad Transition, I built that dashboard for wages and jobs. The safety world has been building its own version, under a different name.

In September 2024, writing for the Carnegie Endowment, Holden Karnofsky laid out if-then commitments for AI risk. The form is: “If an AI model has capability X, risk mitigations Y must be in place.” And if they can’t be in place in time, delay.

His best line is the reason this works across the whole debate. Adopting them “does not require agreement on whether major AI risks are imminent,” only that certain situations would call for certain responses “if they came to pass.”

That’s how a doomer and an accelerationist can sign the same piece of paper.

So I’d run three dials side by side. The thresholds below are mine and illustrative, not anyone’s official proposal.

1. The wage dial. Labor’s share of income, long unemployment spells, and median wages. When they break, speed up the gift-back and universal basic capital. That’s Part 19’s dashboard.
2. The capability dial. Tripwire abilities, like a model that can walk a novice through building a weapon of mass destruction, which is Karnofsky’s own example. Plus the speed of progress itself, measured below. When a tripwire trips, the mitigations go in, or development waits.
3. The concentration dial. How much frontier compute and how much of the robot fleet sits with the top few owners, and whether the military buys AI from more than one provider. When concentration rises, speed up compute cooperatives and the gift-back.

The speed of progress is measurable. In March 2025, the research group METR found that the length of tasks AI agents can finish had been doubling about every 7 months for six years.

If that doubling time suddenly shortens, that’s foom showing up in the data, and the brake comes out.

There’s even a ready-made test for which takeoff we’re in.

In 2018, the AI researcher Paul Christiano defined a slow takeoff this way: “There will be a complete 4 year interval in which world output doubles, before the first 1 year interval in which world output doubles.”

A four-year doubling works out to roughly 19% growth a year. If we see that, we’re in a slow takeoff, by Christiano’s definition. If we see a one-year doubling first, it’s fast, and every response on the board should already be law.

Notice that none of this needs anyone to win the argument first.

The honest limits

I promised to be honest about the limits. Here are three.

A forest is harder to watch. In Your First Party Member Should Be an AI You Own, I admitted that billions of personally owned AIs are harder to oversee than a few in data centers.

That’s still true. Local by default means some risks get caught later.

Hanson made a point that cuts against me. In the same 2008 debate, he argued that “the less local is manufacturing, the harder it will be” for a rogue AI to quietly build the equipment it needs. Every region with its own fab means more fabs to watch.

My best answer is the second ledger. Resource accounts log what every fab makes, in physical units, and flag what doesn’t add up. That’s monitoring of flows, not people.

But Yudkowsky’s reply to Hanson still stands. A smart enough system, he wrote, “should have no trouble coming up with good excuses or bribes to cover nearly anything it wants to do.” No ledger is proof against that.

Towns can’t sign treaties. If the capability dial says stop, the brake gets pulled by nations, not trusts.

Worldbuilders should be ready to back it. A steering wheel doesn’t replace the brakes. Every car has both.

Objections

Isn’t spreading out the machines just proliferation with better branding?

For the most dangerous capabilities, yes, which is why the capability dial exists. But most of what this series puts in towns is robots for farms, clinics and power plants, plus assistants on your laptop. Those spread power, not doom.

The frontier can be declared and inspected. The everyday fleet should be owned.

If the doomers are right, isn’t this rearranging deck chairs?

If they’re right and the brake never comes, maybe. But if they’re right, the brake has to be agreed to, enforced for years, and eventually lifted. Every one of those steps goes better when no small group gets to keep the future for itself.

Won’t slower, shared ownership lose the race?

To whom, and for what? AI 2027‘s own happy ending shows what winning looks like: a committee of ten to twenty people.

I’d rather win the way America won with GPS, as I described in Part 10: build it, then make it everyone’s.

Brake, gas, wheel

The doomers are right that we might not get a second try. The accelerationists are right that delay costs lives.

Both are arguing about a pedal. Neither tells you who’s driving.

Here’s what I’d build either way.

Machines owned by many towns, not a handful of labs. Consumer AI that lives on your own device. Universal basic capital, so the gains land with everyone.

The human-will rule, so people stay in the loop. And triggers, so we act on what’s happening instead of what anyone predicted.

If the takeoff is slow, that’s how we share it. If it’s fast, that’s how we keep anyone from seizing it. If it’s deadly, that’s what makes the brake easier to agree on, and safer to release.

Same design. Three futures.

Doomers want a brake. Accelerationists want a gas pedal. Worldbuilders want a steering wheel. And we want a lot of hands on it.

The concentration dial deserves more work than I can give it here: how to measure who owns the machines, town by town, while there’s still time to change it. If you’re working on that, I’d love to compare notes.

That’s the last part of The World We Want. For the full design behind it, with the numbers and the sources, read the whitepaper.

If you’re new here, start with Part 1: Machines Can Make Us Both Free and Safe. This piece is that title, tested against the worst case.