The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
ElevenLabs' AI voice cloning software was misused by 4chan users to generate deepfake audio impersonating celebrities, spreading racist, sexist, and violent content, and enabling fraud. The incidents caused reputational harm and raised concerns about AI-driven identity theft and abuse, prompting the company to consider additional safeguards.[AI generated]
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (voice synthesis and voice cloning AI) that was used maliciously to produce harmful deepfake audio content spreading hate speech. This constitutes a violation of rights and harm to communities, fulfilling the criteria for an AI Incident. The harm is realized as the hateful content was actively generated and disseminated. The company's mitigation measures are complementary information but do not negate the incident classification.[AI generated]
