AI-Generated Personas Used in Sextortion Scams Lead to Suicide and Police Investigations

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Criminals used AI-generated personas and images in sextortion scams, deceiving victims into sharing intimate content and then blackmailing them. In Auckland, a 41-year-old man died by suicide after being targeted, prompting a coroner's warning. Police in Kentucky are also investigating similar AI-enabled sextortion targeting teens.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system's involvement is explicit in the creation of the AI-generated persona used for sextortion. The harm is direct and severe: the victim's suicide following blackmail. The AI system was used maliciously to deceive and extort, leading to a fatal outcome. This fits the definition of an AI Incident as the AI system's use directly led to injury and harm to a person. The event is not a hazard since harm has already occurred, nor is it complementary information or unrelated. It is not a beneficial use since the AI was used to cause harm.[AI generated]
AI principles
Respect of human rightsPrivacy & data governance

Industries
Digital security

Affected stakeholders
General publicChildren

Harm types
Physical (death)Psychological

AI system task:
Content generation


Articles about this incident or hazard