Xiaomi SU7 Introduces AI Defense Against External Voice Commands

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Xiaomi will deploy an AI-powered “outside-wake defense” in its SU7 model via August OTA, blocking external wake-up of its Xiao Ai voice assistant when parked with doors and windows closed. The feature, based on Xiaomi’s proprietary voice model, inhibits malicious commands to unlock windows or trunks, reducing unauthorized access risks.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system (Xiaomi's voice assistant 'Xiao Ai') is involved, specifically its voice recognition and control capabilities. The issue relates to the use of the AI system (voice control) being exploited maliciously from outside the car to unlock windows, which could lead to harm such as unauthorized access or theft. Although no harm is reported as having occurred, the vulnerability represents a plausible risk of harm if exploited. Xiaomi's planned OTA update aims to prevent this. Therefore, this event is best classified as an AI Hazard, as it concerns a plausible future harm from the AI system's use that is being addressed.[AI generated]
Industries
Mobility and autonomous vehiclesConsumer productsDigital security
Severity
AI hazard
AI system task:
Recognition/object detectionEvent/anomaly detectionInteraction support/chatbots

Articles about this incident or hazard

Thumbnail Image

曝车外能语音唤醒小爱同学解锁车窗 小米汽车：8月份OTA升级防御功能

2024-07-20
凤凰网（凤凰新媒体）
Why's our monitor labelling this an incident or hazard?
An AI system (Xiaomi's voice assistant 'Xiao Ai') is involved, specifically its voice recognition and control capabilities. The issue relates to the use of the AI system (voice control) being exploited maliciously from outside the car to unlock windows, which could lead to harm such as unauthorized access or theft. Although no harm is reported as having occurred, the vulnerability represents a plausible risk of harm if exploited. Xiaomi's planned OTA update aims to prevent this. Therefore, this event is best classified as an AI Hazard, as it concerns a plausible future harm from the AI system's use that is being addressed.
Thumbnail Image

声音大模型首次上车，小米 SU7"车外唤醒防御"即将上线_手机网易网

2024-07-19
m.163.com
Why's our monitor labelling this an incident or hazard?
The article describes the integration of an AI voice model in a vehicle to improve security by restricting voice commands from outside the car. There is no indication of any harm occurring or any incident caused by the AI system. Instead, it is a planned feature rollout aimed at preventing potential misuse or hazards related to voice control. The mention of prior concerns about voice wake-up security is context for the new feature, which is a mitigation measure. Therefore, this is Complementary Information about an AI system deployment and its security enhancement, not an incident or hazard.
Thumbnail Image

车外能唤醒小爱同学解锁车窗？小米官方：将上线"车外唤醒防御"功能

2024-07-20
环球网
Why's our monitor labelling this an incident or hazard?
An AI system (the voice assistant 'Xiao Ai') is involved, and its misuse (malicious voice commands from outside the car) could lead to harm such as unauthorized unlocking of car windows or trunks, which is a security and property harm. However, the article describes the vulnerability and the company's mitigation measures before any reported incident of harm. Therefore, this event represents a plausible risk of harm that is being addressed, fitting the definition of an AI Hazard rather than an Incident or Complementary Information.