Google AI Search Promotes Malware and Scam Links to Users

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Google's AI-powered Search Generative Experience (SGE) has been found to surface malicious and scam-laden links in search results, exposing users to malware, fraudulent giveaways, and unwanted ads. SEO experts and cybersecurity researchers identified that the AI system is being exploited by scammers, prompting Google to implement ongoing updates to mitigate these harms.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system (Google's SGE) is actively used and its outputs are directly linked to users being directed to harmful malicious sites. This has caused realized harm in the form of scams, phishing, and unwanted software installations, which fits the definition of an AI Incident. The harm is not just potential but occurring, as users are being exposed to malicious content due to the AI's recommendations. Therefore, this event qualifies as an AI Incident.[AI generated]
AI principles
AccountabilityRobustness & digital securitySafetyTransparency & explainability

Industries
Digital securityConsumer services

Affected stakeholders
Consumers

Harm types
Economic/Property

Business function:
Other

AI system task:
Organisation/recommenders


Articles about this incident or hazard