AI-Generated Pornography and Illegal Content Distribution Chain Exposed in China

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Multiple investigations reveal a widespread illegal industry in China using AI to generate and distribute pornographic content, including deepfake videos and explicit chat software. Tutorials and tools are openly sold online, enabling mass production and evasion of regulation, causing harm to individuals and exposing minors to inappropriate material.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions AI technology being used to generate large-scale illegal pornographic videos and content, facilitated by tutorials that teach users how to produce and evade detection. This constitutes a direct use of AI systems leading to violations of applicable laws and harm to communities through the spread of illegal and harmful content. The AI system's role is pivotal in enabling the creation and distribution of this content at scale, fulfilling the criteria for an AI Incident under violations of law and harm to communities. Therefore, this event is classified as an AI Incident.[AI generated]
AI principles
Respect of human rightsPrivacy & data governance

Industries
Media, social platforms, and marketing

Affected stakeholders
ChildrenGeneral public

Harm types
PsychologicalHuman or fundamental rightsReputational

Business function:
Other

AI system task:
Content generation


Articles about this incident or hazard