AI Deepfake Technologies Enable Fraud, Privacy Violations, and Harassment in China

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI-powered face-swapping and voice synthesis technologies have been misused in China for telecom fraud, impersonation scams, and the creation of deepfake pornography, leading to significant financial losses, privacy violations, defamation, and psychological harm to ordinary individuals. Despite legal frameworks, enforcement and public awareness remain insufficient to curb these AI-driven incidents.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an AI system explicitly described as an 'AI video face-swapping' tool that uses AI technology to replace faces in videos. The unauthorized use of the influencer's face constitutes a violation of her portrait rights, a form of human rights infringement under applicable law. The court ruling confirms that the AI system's use directly led to this harm. Therefore, this qualifies as an AI Incident due to the realized violation of rights caused by the AI system's use.[AI generated]
AI principles
AccountabilityPrivacy & data governanceRespect of human rightsRobustness & digital securitySafetyTransparency & explainabilityHuman wellbeing

Industries
Digital securityMedia, social platforms, and marketingFinancial and insurance servicesConsumer services

Affected stakeholders
General public

Harm types
Economic/PropertyReputationalPsychologicalHuman or fundamental rights

AI system task:
Content generation


Articles about this incident or hazard