Twitter Suspends Accounts for AI-Generated Racist Deepfake of 'The Little Mermaid' Actress

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Twitter suspended multiple accounts after users used AI to create and share deepfake images and videos that altered the race of Halle Bailey, the Black actress starring in 'The Little Mermaid,' to make her appear white. The AI-generated content sparked widespread criticism for racism and led to platform intervention.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event explicitly involves the use of AI to create a deepfake video that changes the race of a character, which is a direct use of AI-generated content. The harm is realized as the content promotes racism and causes social harm, including violation of rights and harm to communities. The suspension of the accounts by Twitter confirms the recognition of this harm. Hence, the AI system's use directly led to harm, fitting the definition of an AI Incident.[AI generated]
AI principles
FairnessRespect of human rightsSafety

Industries
Media, social platforms, and marketing

Affected stakeholders
Women

Harm types
ReputationalPsychologicalHuman or fundamental rights

Severity
AI incident

AI system task:
Content generation

In other databases

Articles about this incident or hazard

Thumbnail Image

Racista é suspenso do Twitter por usar IA para embranquecer Pequena Sereia

2022-09-13
uol.com.br
Why's our monitor labelling this an incident or hazard?
The event explicitly involves the use of AI to create a deepfake video that changes the race of a character, which is a direct use of AI-generated content. The harm is realized as the content promotes racism and causes social harm, including violation of rights and harm to communities. The suspension of the accounts by Twitter confirms the recognition of this harm. Hence, the AI system's use directly led to harm, fitting the definition of an AI Incident.
Thumbnail Image

Internauta tem conta suspensa ao transformar 'Pequena Sereia' negra em branca e ruiva

2022-09-14
Quem
Why's our monitor labelling this an incident or hazard?
An AI system was explicitly used to generate a deepfake video altering the race and appearance of Halle Bailey, which led to racist attacks and social harm. The AI-generated content directly contributed to violations of rights and harm to communities by promoting racist narratives and harassment. The suspension of the user's account indicates recognition of the harm caused. Therefore, this qualifies as an AI Incident due to realized harm stemming from the use of AI-generated manipulated content for racist purposes.
Thumbnail Image

'A Pequena Sereia': Twitter suspende perfil que deixou atriz negra com pele branca - 14/09/2022 - Cinema e Séries - F5

2022-09-14
F5
Why's our monitor labelling this an incident or hazard?
An AI system was explicitly used to modify the image of Halle Bailey, changing her skin color and hair, which is a form of AI-generated manipulated content. This manipulation led to social harm by promoting a racially discriminatory portrayal and caused reputational damage to the actress and the film's representation. The event includes direct harm to communities through misrepresentation and racial bias, fulfilling the criteria for an AI Incident. The suspensions of the involved Twitter accounts further indicate the harm was recognized and acted upon. Therefore, this event qualifies as an AI Incident due to the realized harm caused by the AI-generated image manipulation.
Thumbnail Image

Twitter suspende usuário por usar IA para embranquecer atriz de "A Pequena Sereia"

2022-09-14
Olhar Digital - O futuro passa primeiro aqui
Why's our monitor labelling this an incident or hazard?
The event explicitly involves the use of AI to create a deepfake video that changes the race of a real person, which is a clear example of AI-generated manipulated content causing harm. The harm here is a violation of human rights through racial discrimination and the spread of racist content. The suspension of the user by Twitter confirms the recognition of harm and policy violation. The AI system's use directly led to social harm and platform enforcement actions, fitting the definition of an AI Incident under violations of human rights.
Thumbnail Image

Twitter suspende perfil que deixou atriz negra de 'A Pequena Sereia' com pele branca

2022-09-14
Notícias ao Minuto Brasil
Why's our monitor labelling this an incident or hazard?
The event describes the use of an AI system to modify an image in a way that changes racial representation, which is a form of harm to communities and a violation of rights related to dignity and representation. The AI-generated content was used maliciously or irresponsibly, leading to social harm and platform intervention. Therefore, this qualifies as an AI Incident due to the realized harm caused by the AI system's use in generating racially altered images that contribute to discrimination and social harm.
Thumbnail Image

Internauta é banido de plataforma após cometer ato racista contra Halle Bailey - OFuxico

2022-09-14
O Fuxico
Why's our monitor labelling this an incident or hazard?
The article describes a user employing AI (deepfake technology) to alter a video of Halle Bailey, accompanied by racist remarks, which constitutes a violation of rights and harm to communities. The AI system's use directly led to the dissemination of racist content, causing harm. Therefore, this qualifies as an AI Incident due to realized harm stemming from AI misuse.
Thumbnail Image

Twitter suspende a usuario que compartió deepfake de "La Sirenita" con Halle Bailey

2022-09-14
RPP noticias
Why's our monitor labelling this an incident or hazard?
The event describes the use of AI to create a deepfake video that changed the skin color, hair, and facial features of Halle Bailey in the trailer of The Little Mermaid. This AI-generated content was shared widely, causing significant social controversy and racial tensions, which constitutes harm to communities. The suspension of the user account indicates a response to this harm. Since the AI system's use directly led to social harm and controversy, this qualifies as an AI Incident under the framework, specifically harm to communities (d).
Thumbnail Image

"La sirenita": Twitter suspende a usuario que publicó tráiler deepfake de Halle Bailey

2022-09-15
LaRepublica.pe
Why's our monitor labelling this an incident or hazard?
The event describes the use of an AI system to create a deepfake video that alters an actress's appearance in a way that has caused social harm and controversy, including racist implications. The AI-generated content has been used maliciously, leading to the suspension of the user. This constitutes harm to communities through the spread of racially offensive manipulated media, which fits the definition of an AI Incident under violations of human rights or harm to communities. The AI system's use directly led to this harm. Therefore, this event qualifies as an AI Incident rather than a hazard or complementary information.