AI-Generated Fake News on Missing Child Incident Leads to Arrests

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A network of individuals used AI to generate and spread false information about a missing 4-year-old girl in Shanghai, causing public misinformation and social harm. The group, linked to a tech company, controlled over 100 accounts to disseminate fake news. Authorities arrested 12 individuals involved in the operation.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system was explicitly used to create a fabricated video with false content, which was then disseminated causing harm through misinformation. This constitutes a violation of rights and harm to communities by spreading false and damaging information. The event describes realized harm caused by the AI-generated content, thus qualifying as an AI Incident.[AI generated]
AI principles
AccountabilityTransparency & explainabilityRobustness & digital securitySafetyHuman wellbeingRespect of human rightsDemocracy & human autonomy

Industries
Media, social platforms, and marketingDigital security

Affected stakeholders
General publicChildren

Harm types
PsychologicalReputationalPublic interestHuman or fundamental rights

Severity
AI incident

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

男子发布虚假视频受严惩

2024-05-21
新华网广东频道
Why's our monitor labelling this an incident or hazard?
An AI system was explicitly used to create a fabricated video with false content, which was then disseminated causing harm through misinformation. This constitutes a violation of rights and harm to communities by spreading false and damaging information. The event describes realized harm caused by the AI-generated content, thus qualifying as an AI Incident.
Thumbnail Image

"女童南汇海滩走失"事件中的网络水军:AI批量造帖、控制百余账号转发_浦江头条_澎湃新闻-The Paper

2024-05-17
The Paper
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems to generate and disseminate false information in large volumes, which directly led to harm to individuals (mental distress to the girl’s family), harm to communities (misleading public opinion, social panic), and disruption of social order. The AI system's role in batch-generating posts and coordinating account activity is pivotal to the incident. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use directly caused significant harm.
Thumbnail Image

案件详情公布!团伙炒作"女童南汇海滩走失"事件,AI批量造帖

2024-05-18
扬子网(扬子晚报)
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems to generate and disseminate false information at scale, which directly caused harm to the mental health of the victim's family and misled the public, constituting harm to communities and individuals. The AI system's role in mass-producing fake posts is pivotal to the incident. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use directly led to violations of rights and harm to communities.
Thumbnail Image

"女童南汇海滩走失"事件中的网络水军:AI批量造帖、控制百余账号转发

2024-05-17
华商网
Why's our monitor labelling this an incident or hazard?
The event explicitly involves the use of AI to batch-create false posts and coordinate a large number of accounts to spread misinformation, which directly led to harm including mental distress to the victim's family, public misjudgment, social panic, and disruption of social order. The AI system's role in generating and amplifying harmful content is pivotal. The harms are realized and documented, with law enforcement intervention confirming the incident's severity. Hence, this is an AI Incident rather than a hazard or complementary information.
Thumbnail Image

"女童南汇海滩走失"事件中的网络水军:AI批量造帖、控制百余账号转发

2024-05-17
163.com
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems to batch-create false posts and coordinate mass dissemination of misinformation, which directly led to harm to individuals (mental distress to the girl and family), harm to communities (social panic and public order disruption), and violations of rights (harassment and defamation). The AI system's role in generating and spreading harmful content is pivotal. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use directly caused significant harm.