Illegal AI Face-Swap Targets Aespa and Other Celebrities

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Malicious AI face-swapping technology was used to generate illicit videos and images of K-pop group aespa, along with Dilraba Dilmurat, Liu Yifei, Yang Mi, JJ Lin, Jin Dong and Taylor Swift. These deepfakes infringe privacy, cause reputational and financial harm, and have triggered lawsuits and calls for stricter AI legislation.[AI generated]

Why's our monitor labelling this an incident or hazard?

The events involve the use of AI face-swapping technology (an AI system) to create unauthorized and harmful content, including scams and defamation, which constitute violations of rights and harm to individuals. The harms are realized, as evidenced by legal actions, financial losses, and public concern. Therefore, these events meet the criteria for AI Incidents because the AI systems' use directly led to violations of rights and harm to persons and communities.[AI generated]
AI principles
AccountabilityPrivacy & data governanceRespect of human rightsSafetyTransparency & explainability

Industries
Media, social platforms, and marketing

Affected stakeholders
General public

Harm types
Human or fundamental rightsReputationalEconomic/Property

Severity
AI incident

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

管理办法已正式施行!遭遇"强制刷脸",这样应对!

2025-06-03
news.cn
Why's our monitor labelling this an incident or hazard?
The article centers on the enactment of new regulations governing facial recognition technology, aiming to prevent misuse and protect personal rights. It does not describe any actual harm or incident caused by AI systems, nor does it report a plausible imminent harm event. The content is primarily about governance, legal frameworks, and public guidance, which fits the definition of Complementary Information as it enhances understanding of AI governance and societal responses without reporting a new AI Incident or Hazard.
Thumbnail Image

aespa遭非法AI换脸引行业震动,恶意内容未涉其他艺人待法律严惩

2025-06-05
k.sina.com.cn
Why's our monitor labelling this an incident or hazard?
The events involve the use of AI face-swapping technology (an AI system) to create unauthorized and harmful content, including scams and defamation, which constitute violations of rights and harm to individuals. The harms are realized, as evidenced by legal actions, financial losses, and public concern. Therefore, these events meet the criteria for AI Incidents because the AI systems' use directly led to violations of rights and harm to persons and communities.
Thumbnail Image

aespa遭非法AI换脸引行业震动,恶意内容未涉其他艺人待法律严惩

2025-06-05
k.sina.com.cn
Why's our monitor labelling this an incident or hazard?
The article explicitly describes the use of AI face-swapping technology to create unauthorized and harmful content involving celebrities, which has led to legal disputes, fraud, and reputational harm. This constitutes a violation of rights and harm to individuals, fitting the definition of an AI Incident. The AI systems' use in generating these deepfake contents is central to the harm caused, thus qualifying the event as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

新规实施!本月起,刷脸不能太任意,违法或被追刑责

2025-06-03
news.bjd.com.cn
Why's our monitor labelling this an incident or hazard?
The article centers on the introduction of a regulatory framework to manage the safe and lawful use of AI-powered facial recognition technology. It discusses measures to prevent privacy violations and unauthorized data use, aiming to mitigate risks before harm occurs. Since it does not describe any realized harm or a specific event where AI caused or could cause harm, it does not qualify as an AI Incident or AI Hazard. Rather, it is a governance response providing complementary information about AI safety and regulation.
Thumbnail Image

不得强制!国家已明确:本月起开始实行新规,违者最高可判刑7年_手机网易网

2025-06-03
m.163.com
Why's our monitor labelling this an incident or hazard?
The article clearly involves AI systems, specifically facial recognition AI, whose use has directly led to harms including privacy violations, identity theft, and fraud. These harms fall under violations of human rights and harm to individuals and communities. The new regulations and penalties are a governance response to these realized harms. Therefore, this event qualifies as an AI Incident because the development and use of facial recognition AI systems have directly caused significant harms, and the article reports on both the harms and the regulatory measures addressing them.