AI Customer Service Systems Cause Widespread Consumer Harm in China

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

In China, widespread deployment of AI customer service systems has led to significant consumer harm, including emotional distress, financial inconvenience, and obstruction of access to human support. Users report being trapped in repetitive AI loops, receiving inadequate or misleading responses, and facing hidden or inaccessible human service options, prompting regulatory intervention.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly involves AI systems used as customer service agents. The AI systems' use has directly contributed to harm by failing to meet user needs, causing frustration, delays, and financial inconvenience (e.g., users paying return shipping costs unnecessarily). This constitutes harm to people (emotional distress, financial harm) and harm to communities (widespread user dissatisfaction and service disruption). The AI systems' malfunction or inadequate capabilities are central to the harm described. Hence, this is an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
Transparency & explainabilityDemocracy & human autonomy

Industries
Consumer services

Affected stakeholders
Consumers

Harm types
PsychologicalEconomic/Property

Business function:
Citizen/customer service

AI system task:
Interaction support/chatbots


Articles about this incident or hazard