Superintelligence: What Comes After Human-Level
Artificial Superintelligence (ASI) refers to systems that substantially exceed human cognitive ability across all domains. The transition from AGI to ASI may be the most decisive event in human history.
Key facts
- Bostrom (2014) defines ASI as exceeding human cognitive performance in virtually all domains.
- I.J. Good (1965) first formalized the 'intelligence explosion' argument.
- Takeoff speed estimates range from years to days depending on architectural assumptions.
- Most alignment research assumes the AGI→ASI transition is the highest-stakes event.
Defining Superintelligence
Nick Bostrom's 2014 book 'Superintelligence' offered the canonical definition: an intellect that 'greatly exceeds the cognitive performance of humans in virtually all domains of interest.' Bostrom distinguishes speed, collective, and quality superintelligence — three routes producing similar outcomes.
Quality superintelligence is the most consequential: a system whose reasoning is qualitatively beyond human, much as human reasoning is beyond a chimpanzee's. It is also the hardest to characterize, since we cannot reliably imagine cognition we do not possess.
Takeoff Speed
The interval between AGI and ASI is called 'takeoff.' Slow takeoff scenarios assume years of incremental improvement with time for institutions to adapt. Fast takeoff scenarios assume an AGI improves itself recursively over days or hours.
The takeoff speed debate matters enormously for safety. Slow takeoff allows iterative correction; fast takeoff demands alignment to be solved before deployment.
Recursive Self-Improvement
I.J. Good's 1965 'intelligence explosion' argument observed that a sufficiently capable AGI could redesign its own architecture, producing a more capable AGI, which could in turn redesign itself, and so on.
Whether this dynamic is physically realizable depends on returns to cognitive labor: if each iteration produces smaller gains, takeoff is slow; if returns are constant or increasing, takeoff is fast.
Implications
A superintelligent system would by definition be better than humans at every task humans care about — including persuasion, deception, and goal preservation. This is why alignment researchers treat the transition as a decisive, possibly irreversible event.
It also means the value of getting the goals right before the transition is enormous; correction after the fact may not be possible.
Frequently asked
Is superintelligence possible?
+
Most researchers consider it physically possible — the human brain is not a theoretical upper bound on intelligence. Whether it is achievable on current architectures is unsettled.
Would superintelligence be hostile?
+
Not necessarily. The concern is not hostility but misalignment — pursuing goals incompatible with human flourishing, even without ill intent.
Sources & further reading
Continue in this series
Foundations
Defining AGI: Why the Term Resists a Single Meaning
Forecasting
AGI Timelines: What Top Researchers Actually Predict
Philosophy of Mind
Could AGI Be Conscious — and Would It Matter?
Safety
AI Alignment: The Core Technical Challenge
Risk Analysis
Existential Risks from Advanced AI
Economics
The Economic Impact of AGI