Qurated: Stories of the future are undermined by agent assumptions
The Hidden Assumption That Breaks Every AI Forecast
Every prediction about AI's future secretly answers a question no one is asking out loud: What is the basic unit of the future?
Get that wrong, and every downstream forecast—about compute, alignment, and governance—inherits the error.
The Ontological Sleight of Hand
We frame our disagreements as technical. Scaling curves. Algorithmic progress. Unipolar versus multipolar power. But these are surface arguments.
Underneath, forecasters smuggle in ontological commitments: untestable assumptions about what the future is made of.
Ask any forecast one question—Who are the players?—and watch the whole model reorganize:
- Humans? Then AI is a tool, and governance means controlling access.
- Institutions? Then AI amplifies existing power, and governance means antitrust and oversight.
- AI models? Then agents are the actors, and governance means alignment.
- Human-AI hybrids? Then the unit is a collective, and governance means something we barely have language for.
Each answer produces a different theory of what goes wrong—and a different theory of how to prevent it.
Why the Disagreements Are Really About This
Consider the live debate. AI 2027 and AI 2040 project confident trajectories. Richard Ngo counters that something is fundamentally off with how LLMs are currently structured as agents.
Notice: this isn't a fight about data. It's a fight about what kind of thing an agent is. Change your model of agency, and you change what to fear, what to build, and what to regulate.
The forecasts are downstream of the ontology. We keep debating the conclusions when the premises are where the disagreement actually lives.
A Framework: Name Your Unit
Before trusting any AI prediction—yours or anyone's—run it through three questions:
1. What is the atomic agent? Is the actor a model, a person, an organization, or a hybrid? Every forecast has an answer, even when unstated. Find it.
2. Does the scenario let that unit change scale? Weak forecasts fix the agent at one level. Strong ones recognize agency operates at every scale—a neuron, a person, a firm, a civilization. If your model only works at one altitude, it's brittle.
3. What breaks if the unit is wrong? Stress-test by swapping the assumed agent. If your governance plan collapses when "the player" shifts from model to hybrid collective, you've found the load-bearing assumption—and its fragility.
The Practical Move: Forecast Under Uncertainty
Here is the paradox. We cannot know the true ontology of future agents—yet we cannot walk into that future blind.
The answer is not to pick one model and commit. It's to build approximate solutions robust across ontologies.
- Prefer governance mechanisms that work whether the agent is human, institutional, or artificial.
- Treat single-ontology confidence as a red flag, not a virtue.
- Track processes rather than agents—the patterns that hold regardless of who or what is executing them.
This is the seed of process alignment: aligning the process by which agency operates at every scale, rather than betting everything on one guess about what agents are.
The Takeaway
The most dangerous assumptions are the ones you never notice you made.
When you read the next confident AI scenario, don't argue with its numbers. Ask what it thinks the world is made of. That single question separates forecasts that merely sound rigorous from those that survive contact with a future we can't yet name.