AI Voice Cloning Used Without Consent in Sexually Explicit Content Harms Voice Actor

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Voice actor Allegra Clark discovered her voice was cloned using AI and inserted without consent into a sexually explicit scene, causing personal and professional harm. The AI company ElevenLabs failed to provide a satisfactory response, highlighting risks of AI misuse in content creation and the lack of effective safeguards.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use and misuse of generative AI voice cloning systems that have directly led to harms including violation of voice actors' rights, harassment through AI-generated offensive content, and economic harm by replacing human voice actors with AI-generated voices without consent or compensation. These constitute violations of human and intellectual property rights and harm to livelihoods, fitting the definition of an AI Incident. The article details realized harms, not just potential risks, and thus is classified as an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
AccountabilityPrivacy & data governanceRespect of human rightsSafetyTransparency & explainabilityHuman wellbeingRobustness & digital security

Industries
Media, social platforms, and marketingArts, entertainment, and recreation

Affected stakeholders
Workers

Harm types
PsychologicalReputationalEconomic/PropertyHuman or fundamental rights

Severity
AI incident

Business function:
Other

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

'Keep Your Paws Off My Voice': Voice Actors Worry Generative AI Will Steal Their Livelihoods

2023-10-09
Forbes
Why's our monitor labelling this an incident or hazard?
The event involves the use and misuse of generative AI voice cloning systems that have directly led to harms including violation of voice actors' rights, harassment through AI-generated offensive content, and economic harm by replacing human voice actors with AI-generated voices without consent or compensation. These constitute violations of human and intellectual property rights and harm to livelihoods, fitting the definition of an AI Incident. The article details realized harms, not just potential risks, and thus is classified as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

An actor who found an AI-generated version of her voice reproduced in a sexually suggestive scene on TikTok said the experience was 'violating'

2023-10-10
Business Insider
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (generative AI voice synthesis by ElevenLabs) that was used without consent to create manipulated voice content that is sexually suggestive. This unauthorized use has caused harm to the voice actor's personal dignity and professional reputation, which constitutes a violation of rights under applicable law (intellectual property and personal rights). Therefore, this qualifies as an AI Incident due to the realized harm caused by the AI system's use.
Thumbnail Image

An actor who found an AI-generated version of her voice reproduced in a sexually suggestive scene on TikTok said the experience was 'violating'

2023-10-10
Business Insider Nederland
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of an AI system (ElevenLabs) to generate a voice clone of the actor without permission, which was then used in a sexually suggestive context. This unauthorized use caused emotional harm and potential reputational damage, constituting a violation of rights. The AI system's role is pivotal as it enabled the creation of the content without consent. The harm is realized, not just potential, and involves violation of personal and intellectual property rights. Hence, this is an AI Incident.
Thumbnail Image

Sesini müstehcen bir sahnede duyunca şoke oldu... Yapay zekâya tepkiler artıyor

2023-10-11
Sözcü Gazetesi
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of a generative AI system to copy and manipulate a voice actor's voice without permission, resulting in the creation of a sexually explicit scene. This unauthorized use of the AI system has caused harm to the individual by misrepresenting her and potentially damaging her reputation and professional relationships. This fits the definition of an AI Incident because the AI system's use directly led to a violation of rights and harm to the person involved.
Thumbnail Image

Google, Yapay Zekâ Araçlarını Kullananları Koruyacak

2023-10-13
Webtekno
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (Google's AI tools) and addresses legal risks related to copyright infringement, which is a violation of intellectual property rights. However, the article does not describe any actual harm or lawsuit occurring yet; it focuses on Google's policy to manage and assume responsibility for such legal risks. Therefore, this is a governance and societal response to potential AI-related legal harms, enhancing understanding of the AI ecosystem and risk management, but it does not report a new AI Incident or AI Hazard itself.
Thumbnail Image

Akıl almaz artı 18 film tuzağı

2023-10-11
Yeni Çağ Gazetesi
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (voice cloning technology) used without consent to create manipulated content that harms the individual (a voice actor). This constitutes a violation of rights and harm to the person, fitting the definition of an AI Incident. The AI system's use directly led to the harm described, and the lack of adequate response from the AI company exacerbates the issue.
Thumbnail Image

Medya yapay zekaya karşı birleşti - Türkiye Gazetesi

2023-10-15
Türkiye
Why's our monitor labelling this an incident or hazard?
The article clearly involves AI systems, specifically generative AI models that produce photorealistic images and videos. These AI systems have the potential to cause harm by spreading misinformation and undermining trust in media, which is a recognized harm to communities. However, the article does not describe a specific event where such harm has occurred or a direct incident caused by AI misuse or malfunction. Instead, it focuses on the formation of initiatives and technological solutions to detect and verify AI-generated content, which are responses to potential or ongoing challenges. Therefore, this is best classified as Complementary Information, as it provides context and details about societal and technical responses to AI-related challenges rather than reporting a new AI Incident or AI Hazard.