Google AI Overviews Reduce Publisher Traffic and Enable Fraud via Fake Contact Info

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Google's AI Overviews have led to significant declines in website traffic for publishers, causing economic harm. Additionally, both Google's and OpenAI's AI systems have been exploited by scammers to display fake customer service numbers, resulting in users being defrauded after trusting AI-generated information.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (Google Gemini and ChatGPT) whose outputs were maliciously manipulated to present false information (fake phone numbers) that users trusted and acted upon, resulting in financial harm (theft). This fits the definition of an AI Incident because the AI systems' use and malfunction (due to prompt injection attacks) directly led to harm to people (financial injury). The article describes realized harm, not just potential risk, and details the causal role of AI in the fraud.[AI generated]
AI principles
AccountabilityFairnessTransparency & explainabilityRobustness & digital securitySafety

Industries
Media, social platforms, and marketingDigital securityConsumer services

Affected stakeholders
BusinessConsumers

Harm types
Economic/Property

AI system task:
Content generation