AI-Generated Content Spurs Fraud; Multimodal Detection System Deployed in China

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI-generated fake images and videos have enabled fraud in e-commerce and financial sectors in China, including identity theft and false claims. In response, the company 合合信息 launched a multimodal AI detection system, now deployed across over 30 industries, significantly improving the interception of deepfakes and reducing related harms.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions an AI system developed to detect AI-generated fake content, which is already deployed in real-world scenarios to prevent and mitigate harms like identity fraud, false claims, and deceptive e-commerce activities. These harms fall under violations of rights and harm to communities. Since the AI system's use is directly linked to preventing and addressing these harms, this qualifies as an AI Incident rather than a hazard or complementary information. The system's deployment and impact on reducing fraud demonstrate realized harm and response to AI-generated threats.[AI generated]
AI principles
Transparency & explainabilityRespect of human rights

Industries
Financial and insurance servicesLogistics, wholesale, and retail

Affected stakeholders
ConsumersBusiness

Harm types
Economic/PropertyHuman or fundamental rights

AI system task:
Content generation


Articles about this incident or hazard