AI Deepfake Technology Used to Create and Distribute Illicit Videos Leads to Legal Action in Hangzhou

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A man in Hangzhou used AI face-swapping technology to create and sell pornographic deepfake videos without consent, including custom orders using celebrity images. He also sold the software and tutorials. Authorities charged him with criminal and civil offenses, citing violations of personal privacy and public information security.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions the use of AI face-swapping (deepfake) technology to produce and spread harmful content without consent, causing violations of personal information rights and public information security. The AI system's use directly caused realized harm to individuals and society, including privacy violations and social harm. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use directly led to violations of rights and harm to communities and public information security.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsSafetyAccountability

Industries
Media, social platforms, and marketing

Affected stakeholders
Other

Harm types
Human or fundamental rightsReputational

Severity
AI incident

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

男子用AI换脸技术侵害公共信息安全 杭州检方提起民事公益诉讼 - cnBeta.COM 移动版

2023-06-20
cnBeta.COM
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of AI face-swapping (deepfake) technology to produce and spread harmful content without consent, causing violations of personal information rights and public information security. The AI system's use directly caused realized harm to individuals and society, including privacy violations and social harm. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use directly led to violations of rights and harm to communities and public information security.
Thumbnail Image

男子用AI换脸技术侵害公共信息安全 杭州检方提起民事公益诉讼

2023-06-20
东方财富网
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (AI face-swapping technology) to produce harmful content (pornographic videos) and distribute it illegally, causing harm to public information security and social public interest. The AI system's use directly led to violations of law and harm to communities by spreading illicit content. Therefore, this qualifies as an AI Incident due to realized harm stemming from the AI system's use.
Thumbnail Image

男子用AI换脸技术侵害公共信息安全 杭州检方提起民事公益诉讼

2023-06-20
东方财富网
Why's our monitor labelling this an incident or hazard?
The event explicitly involves the use of an AI system (AI face-swapping/deepfake technology) to produce harmful content (non-consensual pornographic videos) and distribute it for profit, which constitutes a violation of personal rights and public information security. The harm is realized and significant, including privacy violations, social harm, and legal consequences. The involvement of AI in the creation and dissemination of harmful content meets the criteria for an AI Incident as the AI system's use directly led to harm to individuals and society.
Thumbnail Image

男子用AI换脸技术侵害公共信息安全 杭州检方提起民事公益诉讼

2023-06-20
凤凰网(凤凰新媒体)
Why's our monitor labelling this an incident or hazard?
The event explicitly involves the use of an AI system (AI face-swapping technology) in the creation and distribution of harmful content without consent, which constitutes a violation of personal rights and public information security. The harms are realized and significant, including infringement on sensitive personal information, social public interest damage, and potential ongoing risks to many individuals. The involvement of AI in the development, use, and misuse of the technology is central to the incident. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use has directly led to harm to individuals and communities, as well as violations of legal rights.
Thumbnail Image

男子用AI换脸技术合成"明星淫秽视频",被提起公诉

2023-06-20
app.myzaker.com
Why's our monitor labelling this an incident or hazard?
The event explicitly involves the use of an AI system (AI face-swapping technology) in the development and use phases to create harmful content without consent, leading to realized harm including violations of personal privacy, reputation, and public information security. The AI system's role is pivotal in producing and distributing the harmful videos and enabling illegal services. The harms include violations of human rights and social public interest, fitting the definition of an AI Incident. The legal actions taken confirm the recognition of actual harm caused by the AI system's misuse.
Thumbnail Image

17:19 男子用AI换脸技术侵害公共信息安全 杭州检方提起民事公益诉讼

2023-06-20
每日经济新闻
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (AI face-swapping technology) to produce harmful content (pornographic videos) and illegal commercial activities, which directly harm public information security and social public interests. The use of AI to create and distribute such content constitutes a violation of rights and harms communities. Therefore, this qualifies as an AI Incident due to realized harm caused by the AI system's use.