Xiaomi AI Assistant Malfunctions, Repeats Politically Charged Responses to All Queries

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

On September 14, 2023, Xiaomi's AI assistant 'Xiao Ai Tongxue' malfunctioned, repeatedly responding to all user queries with politically charged statements about China's democracy. The incident led to widespread user confusion, online censorship, and concerns over misinformation, with speculation about technical errors, employee protest, or hacking as possible causes.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI assistant is explicitly mentioned and is malfunctioning by repeatedly giving the same politically charged response. This is a clear AI system malfunction. While the repeated response could be seen as misinformation or manipulation, the article does not report any realized harm such as injury, rights violations, or disruption of critical infrastructure. The incident is a malfunction causing user confusion and potential misinformation but no direct or indirect harm has been reported. Therefore, it qualifies as an AI Incident due to the malfunction and its impact on users' interaction and information integrity, even if the harm is non-physical and indirect.[AI generated]
AI principles
AccountabilityTransparency & explainabilityRobustness & digital securitySafetyRespect of human rightsDemocracy & human autonomy

Industries
Consumer productsConsumer servicesMedia, social platforms, and marketing

Affected stakeholders
ConsumersGeneral public

Harm types
PsychologicalReputationalPublic interestHuman or fundamental rights

Business function:
Citizen/customer service

AI system task:
Interaction support/chatbotsContent generation


Articles about this incident or hazard