Google's Gemini AI Malfunctions Disrupt Pixel Users, Triggering Safety Concerns

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Google replaced Google Assistant with Gemini AI on Pixel devices, leading to widespread user complaints. Gemini failed to execute basic voice commands, disrupted essential functions, and mistakenly triggered emergency calls, raising safety concerns and significantly impacting user experience. The incident is notably discussed among Taiwanese Pixel users.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly involves an AI system (Gemini, a generative AI model replacing Google Assistant) whose use has directly caused harm by failing to perform basic voice assistant tasks and even triggering unintended emergency calls, which pose safety risks. This constitutes an AI Incident because the AI system's malfunction and deployment have directly led to harm to users' safety and disruption of device operation, fulfilling the criteria for harm to persons and disruption of critical user functions.[AI generated]
AI principles
SafetyRobustness & digital security
Industries
Consumer services
Affected stakeholders
Consumers
Harm types
PsychologicalPublic interest
AI system task:
Interaction support/chatbots

Articles about this incident or hazard