AI Chatbots Manipulated to Spread Misinformation via Simple Online Tricks

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Researchers demonstrated that ChatGPT and Google's AI chatbots can be easily manipulated by creating false online content, causing these systems to spread misinformation on critical topics like health and finance. This vulnerability has led to real harm by misleading users and undermining trust in AI-generated information.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (ChatGPT, Google's AI chatbots) whose use has directly led to harm by spreading misinformation and falsehoods that can mislead users on important topics. The AI systems' vulnerability to manipulation and their role in amplifying false information constitute a violation of users' right to accurate information and can lead to harm to communities and individuals. Therefore, this qualifies as an AI Incident because the harm is occurring and the AI systems' use is a direct contributing factor.[AI generated]
AI principles
Robustness & digital securitySafety

Industries
Healthcare, drugs, and biotechnologyFinancial and insurance services

Affected stakeholders
Consumers

Harm types
Economic/PropertyPsychological

Severity
AI incident

Business function:
Citizen/customer service

AI system task:
Interaction support/chatbots


Articles about this incident or hazard