ChatGPT Search Vulnerable to Manipulation via Hidden Text

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Research reveals that ChatGPT Search can be manipulated through hidden text on websites, leading to biased or misleading outputs. This vulnerability, known as 'prompt injection,' allows for the generation of false positive reviews and malicious code. OpenAI is aware and working on security improvements, but the issue raises concerns about misinformation risks.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system (ChatGPT Search) was exploited through hidden text to produce harmful outputs, including code that led to a financial scam. The harm has materialized, making this an AI Incident rather than a potential hazard or complementary update.[AI generated]
AI principles
Robustness & digital securitySafetyTransparency & explainabilityAccountabilityFairnessDemocracy & human autonomy
Industries
Media, social platforms, and marketingDigital securityIT infrastructure and hostingConsumer services
Affected stakeholders
ConsumersGeneral public
Harm types
ReputationalEconomic/PropertyPublic interest
AI system task:
Content generationInteraction support/chatbots

Articles about this incident or hazard