AI Voice Cloning and Scam Ads Lead to Financial Harm in China and Thailand

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

In China, scammers used AI voice cloning and voice-changing software to impersonate victims' relatives, leading to significant financial losses, including a case where 4.3 million yuan was stolen in 10 minutes. In Thailand, the Consumer Council plans to sue Meta for failing to block scam ads on Facebook, resulting in widespread consumer harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the malicious use of AI systems to replicate human voices for fraudulent purposes, directly leading to financial harm to individuals (a form of harm to persons). The AI system's use in voice cloning is central to the scam and the resulting harm. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to realized harm (financial loss) through deception and fraud.[AI generated]
AI principles
AccountabilityRobustness & digital security
Industries
Digital securityMedia, social platforms, and marketing
Affected stakeholders
Consumers
Harm types
Economic/Property
Business function:
Marketing and advertisement
AI system task:
Content generationOrganisation/recommenders

Articles about this incident or hazard