AGI Governance: International Coordination and Oversight
AGI development is concentrating in a small number of well-resourced labs across the US, UK, China, and EU. Coordinating their behavior is among the most consequential governance challenges of the century.
Key facts
- EU AI Act (2024) classifies AI systems by risk tier; high-risk systems face strict requirements.
- AI Safety Institutes now exist in the UK, US, Japan, Singapore, France, and Canada.
- Frontier AI training depends on chips concentrated among TSMC, Samsung, and a few designers.
- No global AI governance body currently has enforcement authority.
Current Regulatory Frameworks
The EU AI Act (2024) is the first comprehensive horizontal AI regulation, classifying systems by risk tier. The US has proceeded through executive orders and sector-specific guidance. The UK established the AI Safety Institute in 2023, followed by similar institutes in the US, Japan, and Singapore.
China's Generative AI regulations (2023–24) impose pre-deployment review and content controls. The patchwork is deepening rather than converging.
Emerging International Bodies
The Bletchley Declaration (2023), Seoul Declaration (2024), and follow-up summits established a multilateral track focused on frontier model safety. Network agreements among AI Safety Institutes now enable joint evaluations.
Proposals for a 'CERN for AI safety' or an 'IAEA for AI' remain conceptual. None has the legal standing or technical capacity to enforce frontier model standards globally.
Compute Governance
Frontier training runs require physical hardware concentrated among a handful of chip designers, fabricators, and cloud providers. This bottleneck makes compute one of the few practically governable inputs to AGI.
US export controls on advanced chips and Chinese investment in domestic fabrication are the highest-stakes compute-governance moves to date. Both have significant unintended consequences.
Open Governance Questions
Whose safety standards count? How are private labs held accountable? What happens when a state actor races ahead? Should AGI development pause at certain capability thresholds? None have settled answers.
The window for establishing durable governance is widely believed to be closing as capabilities advance and economic incentives accelerate.
Frequently asked
Can AI be effectively regulated?
+
Sector-by-sector, yes. Globally and comprehensively, only partially — and only if major powers coordinate, which is structurally difficult.
Why focus on compute?
+
Compute is physically auditable in ways that algorithms and data are not, making it the most practical governance lever for frontier training runs.
Sources & further reading
Continue in this series
Foundations
Defining AGI: Why the Term Resists a Single Meaning
Forecasting
AGI Timelines: What Top Researchers Actually Predict
Beyond AGI
Superintelligence: What Comes After Human-Level
Philosophy of Mind
Could AGI Be Conscious — and Would It Matter?
Safety
AI Alignment: The Core Technical Challenge
Risk Analysis
Existential Risks from Advanced AI