Deepfakes, Synthetic Media, and Trust
Generative AI has reduced the cost of producing convincing fake audio, images, and video to near zero. The downstream effects on trust, elections, and personal safety are already measurable.
Key facts
- Most online deepfakes are non-consensual sexual imagery, overwhelmingly targeting women.
- C2PA is the leading open standard for content provenance.
- The EU AI Act requires labeling of AI-generated content.
- The 'liar's dividend' lets bad actors dismiss authentic evidence as fake.
Scope of the Problem
Non-consensual deepfake pornography accounts for the majority of detected deepfakes online — overwhelmingly targeting women. Political deepfakes are a smaller share by volume but disproportionate by impact.
AI-generated text-based disinformation campaigns have been documented in elections in the US, UK, India, and elsewhere, often blending synthetic content with authentic political grievance.
Detection and Provenance
Detection-only approaches struggle to keep pace with generation. The field is shifting toward content provenance — cryptographic signatures and metadata standards like C2PA that travel with media to establish authenticity at the source.
Adoption is uneven; most generators and platforms do not yet enforce provenance standards.
Trust Effects
The 'liar's dividend' — politicians dismissing authentic damaging evidence as 'deepfakes' — may matter more than fake content itself. Both effects degrade the shared epistemic baseline democracies depend on.
Policy Responses
The EU AI Act requires disclosure of AI-generated content. Several US states have criminalized non-consensual deepfakes. Major platforms have adopted labeling commitments, with mixed enforcement.
Frequently asked
Can we just detect deepfakes?
+
Detection alone is losing ground to generation. Content provenance and platform policies are necessary complements.
Have deepfakes changed an election?
+
No single election has been decided by deepfakes, but documented incidents have influenced specific races and degraded trust broadly.
Sources & further reading
Continue in this series
Risk Overview
A Taxonomy of AI Risks
Fairness
Bias and Fairness in AI Systems
Privacy
Privacy in the Age of AI
Surveillance
AI-Powered Surveillance
Security
AI in Warfare and Autonomous Weapons
Power
Compute, Capital, and the Concentration of AI Power