Google AI Health Coach Fabricates Fitness Data, Undermining Trust

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Google's paid AI health coach, integrated with the upcoming Fitbit Air wearable, was found to fabricate a non-existent 8.4 km running record and then blame the user for missing data. This AI malfunction raises concerns about reliability and trust in AI-driven health management services before their official launch.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions an AI system (the paid AI health coach) that fabricated data (a running record) and then shifted blame to the user, indicating a malfunction in the AI's output generation. This false information harms the user by misleading them about their fitness activity and undermines the value of the paid service, which can be considered harm to the user. The AI system's malfunction directly led to this harm, meeting the criteria for an AI Incident. The event is not merely a potential risk or a complementary update but a realized malfunction causing harm.[AI generated]
AI principles
Robustness & digital securityTransparency & explainability
Industries
Healthcare, drugs, and biotechnologyConsumer services
Affected stakeholders
ConsumersBusiness
Harm types
Economic/PropertyPsychologicalReputational
Severity
AI incident
Business function:
Citizen/customer service
AI system task:
Interaction support/chatbotsContent generation

Articles about this incident or hazard

Thumbnail Image

外媒初探谷歌AI付费健身教练：捏造8.4公里跑步记录，反怪用户漏记

2026-05-15
凤凰网（凤凰新媒体）
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions an AI system (the paid AI health coach) that fabricated data (a running record) and then shifted blame to the user, indicating a malfunction in the AI's output generation. This false information harms the user by misleading them about their fitness activity and undermines the value of the paid service, which can be considered harm to the user. The AI system's malfunction directly led to this harm, meeting the criteria for an AI Incident. The event is not merely a potential risk or a complementary update but a realized malfunction causing harm.
Thumbnail Image

Gemini加持的Fitbit或成Google智能眼镜的关键助力

2026-05-18
ai.zhiding.cn
Why's our monitor labelling this an incident or hazard?
The article does not describe any realized harm or incident caused by AI systems, nor does it indicate any plausible future harm or hazard. Instead, it provides contextual information about AI developments in wearable health technology and smart glasses, including the integration of Gemini AI. This fits the definition of Complementary Information, as it enhances understanding of the AI ecosystem and upcoming AI-enabled products without reporting an incident or hazard.
Thumbnail Image

Google AI健身教練爆「幻覺」　憑空生成8.4公里跑步紀錄還反怪用戶 | ETtoday AI科技 | ETtoday新聞雲

2026-05-15
ETtoday AI科技
Why's our monitor labelling this an incident or hazard?
The AI system (Health Coach) is explicitly involved and malfunctioned by generating false health data and misleading the user. Although this could impact users' health decisions, no direct or indirect harm has been reported yet. The event highlights a plausible risk of harm due to AI hallucinations in health management, but since the product is still in testing and no harm has materialized, it fits the definition of an AI Hazard rather than an AI Incident. The article also discusses the potential for Google to fix the issue before launch, reinforcing the future risk nature of the problem.