Google AI Overviews Directs Users to Scam Phone Numbers, Leading to Financial Losses

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Google's AI Overviews feature has repeatedly displayed fraudulent customer support numbers in search results, leading users to call scammers posing as legitimate representatives. Victims have reported financial losses and unauthorized charges after sharing sensitive information, highlighting the risks of unverified AI-generated content in critical search queries.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system (Google's AI Overviews) is explicitly involved as it generates summaries that include scam phone numbers. The use of these AI-generated summaries has directly led to harm (financial fraud and unauthorized charges), fulfilling the criteria for an AI Incident. The harm is to individuals (harm to persons) through deception and financial loss. Therefore, this event qualifies as an AI Incident due to the direct link between the AI system's outputs and realized harm.[AI generated]
AI principles
AccountabilityRobustness & digital securitySafetyTransparency & explainabilityRespect of human rights
Industries
Consumer services
Affected stakeholders
Consumers
Harm types
Economic/PropertyHuman or fundamental rights
Severity
AI incident
Business function:
Other
AI system task:
Content generation
In other databases
Open in  AIID Logo

Articles about this incident or hazard