AI-Generated Fake Female Profiles Used in Hotel Social App Scam Busted by Shanghai Police

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Shanghai police dismantled a fraud ring behind the 'Quzhu' app, which used AI-driven software to create over 10,000 fake female profiles and generate seductive content. The scam lured users into paying for chat services, resulting in financial losses. Ten suspects were arrested, and the app's fraudulent use of AI was central to the scheme.[AI generated]

Why's our monitor labelling this an incident or hazard?

The app uses intelligent software to batch-create virtual female user accounts and automatically post suggestive content, which is a clear use of AI systems for generating content and managing user interactions. This AI involvement directly causes harm by enabling a fraudulent scheme that financially exploits users. The harm is realized and documented by police investigations and arrests, fulfilling the criteria for an AI Incident due to direct harm (financial fraud) caused by the AI system's use in the scam.[AI generated]
AI principles
AccountabilityHuman wellbeingTransparency & explainabilityDemocracy & human autonomy

Industries
Consumer services

Affected stakeholders
Consumers

Harm types
Economic/Property

Business function:
Citizen/customer service

AI system task:
Content generationInteraction support/chatbots


Articles about this incident or hazard