AI Deepfake Technology Used to Create and Sell Non-Consensual Pornographic Videos in China

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

An Anhui man, Yu, used AI face-swapping technology to create and sell pornographic videos featuring the likenesses of female celebrities, distributing them via online groups for profit. Over 1,200 videos and 1,600 images were seized. Yu was prosecuted for producing and distributing obscene materials, highlighting AI's role in privacy and rights violations.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (AI face-swapping and image manipulation apps) whose malicious use has directly led to harms including violations of personal rights (privacy, portrait rights, reputation), potential criminal offenses, and social harm. The article references actual incidents and legal rulings addressing these harms, indicating realized harm rather than just potential risk. Therefore, this qualifies as an AI Incident. The discussion of regulatory and legal responses is complementary but the primary focus is on the harms caused by malicious AI use.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsSafetyHuman wellbeingAccountabilityTransparency & explainabilityRobustness & digital security

Industries
Media, social platforms, and marketingDigital security

Affected stakeholders
General public

Harm types
Human or fundamental rightsReputationalPsychological

Business function:
Other

AI system task:
Content generationRecognition/object detection


Articles about this incident or hazard