Privacy in the Age of AI
AI is fundamentally a data technology. Its capability scales with data, which puts privacy in direct structural tension with capability — a tension that requires both technical and legal responses.
Key facts
- Differential privacy provides formal mathematical privacy guarantees.
- Federated learning is now used in production by Apple, Google, and others.
- GDPR Article 22 grants rights against solely automated decisions.
- Frontier LLMs have been shown to memorize and reproduce training data verbatim.
The Privacy Threat Model
AI systems amplify three privacy threats: inference (deriving sensitive attributes from non-sensitive data), aggregation (combining data sources beyond what any single source justifies), and re-identification (linking anonymized data to individuals).
Frontier models trained on web-scale corpora may memorize and reproduce personal data, raising questions about consent at training time.
Privacy-Preserving Techniques
Differential privacy adds calibrated noise to provide formal guarantees against membership inference. Federated learning keeps raw data on user devices while sharing model updates. Homomorphic encryption allows computation on encrypted data.
On-device inference — running models locally rather than in the cloud — increasingly enables capable AI without transferring sensitive data.
Privacy Regulation
GDPR (EU) and CCPA (California) establish baseline consent and access rights. Specific AI rules — including data subject rights against automated decisions — are being elaborated in the EU AI Act and state-level US legislation.
Enforcement remains uneven, and the gap between rights on paper and rights in practice is wide.
Frequently asked
Can AI be trained without violating privacy?
+
Yes, in principle — with differential privacy, federated learning, and on-device techniques. Doing so at frontier scale remains an active research area.
Is GDPR enough?
+
It is a strong baseline but does not address all AI-specific concerns. The EU AI Act and emerging AI-specific rules supplement it.
Sources & further reading
Continue in this series
Risk Overview
A Taxonomy of AI Risks
Fairness
Bias and Fairness in AI Systems
Information Integrity
Deepfakes, Synthetic Media, and Trust
Surveillance
AI-Powered Surveillance
Security
AI in Warfare and Autonomous Weapons
Power
Compute, Capital, and the Concentration of AI Power