AI Deepfake Technology Used to Create and Distribute Illicit Videos Leads to Legal Action in Hangzhou

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A man in Hangzhou used AI face-swapping technology to create and sell pornographic deepfake videos without consent, including custom orders using celebrity images. He also sold the software and tutorials. Authorities charged him with criminal and civil offenses, citing violations of personal privacy and public information security.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions the use of AI face-swapping (deepfake) technology to produce and spread harmful content without consent, causing violations of personal information rights and public information security. The AI system's use directly caused realized harm to individuals and society, including privacy violations and social harm. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use directly led to violations of rights and harm to communities and public information security.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsSafetyAccountability

Industries
Media, social platforms, and marketing

Affected stakeholders
Other

Harm types
Human or fundamental rightsReputational

AI system task:
Content generation


Articles about this incident or hazard