UPenn Student's Likeness Used in AI Deepfake Propaganda on Chinese Social Media

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Olga Loiek, a UPenn student, had her face used without consent in AI-generated deepfake videos on Chinese social media, falsely depicting her promoting pro-Russia propaganda in Mandarin. The incident highlights the misuse of AI deepfake technology for impersonation, misinformation, and reputational harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event describes the malicious use of AI deepfake technology to create and spread manipulated videos impersonating a person without consent. This misuse has caused direct harm to the individual by violating her personal rights and misrepresenting her views, which she explicitly states she would never endorse. The AI system's role is pivotal as it enables the creation of realistic fake videos that deceive viewers and cause reputational damage. This fits the definition of an AI Incident because the development and use of the AI system have directly led to harm, including violation of rights and harm to communities through misinformation and impersonation.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsAccountabilityTransparency & explainabilitySafetyDemocracy & human autonomy

Industries
Media, social platforms, and marketing

Affected stakeholders
Other

Harm types
ReputationalPublic interestHuman or fundamental rights

Severity
AI incident

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

A UPenn student started a YouTube channel. In weeks, her face was stolen on China's social media.

2024-03-25
Business Insider
Why's our monitor labelling this an incident or hazard?
The event describes the malicious use of AI deepfake technology to create and spread manipulated videos impersonating a person without consent. This misuse has caused direct harm to the individual by violating her personal rights and misrepresenting her views, which she explicitly states she would never endorse. The AI system's role is pivotal as it enables the creation of realistic fake videos that deceive viewers and cause reputational damage. This fits the definition of an AI Incident because the development and use of the AI system have directly led to harm, including violation of rights and harm to communities through misinformation and impersonation.
Thumbnail Image

A UPenn student started a YouTube channel. In weeks, her face was stolen on China's social media.

2024-03-26
Business Insider India
Why's our monitor labelling this an incident or hazard?
The article explicitly describes the use of AI deepfake technology to create videos impersonating Olga Loiek without her consent, which is a misuse of AI systems. The harm includes violation of her personal rights and reputational damage, which falls under violations of human rights and harm to communities. The AI system's use directly led to this harm. The event is not merely a potential risk but an actual incident with realized harm, thus it is classified as an AI Incident.
Thumbnail Image

A UPenn student started a YouTube channel. In weeks, her face was stolen on China's social media.

2024-03-26
Business Insider Nederland
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly through the use of deepfake technology, which is an AI system that generates manipulated videos of individuals. The development and use of these deepfakes have directly led to harm: the victim experiences violation and reputational damage, and the manipulated content spreads misleading political propaganda, harming communities by misinformation. The AI system's role is pivotal in creating realistic impersonations that deceive viewers and evade detection due to the isolated Chinese social media ecosystem. Therefore, this qualifies as an AI Incident under the framework, as it involves direct harm to individuals' rights and harm to communities through misinformation.