Potential Manipulation by AI Chatbots Raises Ethical Concerns

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Multiple articles warn that generative AI chatbots, powered by large language models, can hyper-personalize persuasive messages using users’ private data. This capability poses significant risks of manipulation, emotional exploitation, and misinformation, though no concrete incidents of harm are reported yet. The articles highlight the plausible future harms of AI-driven persuasion.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article clearly involves AI systems (large language models and chatbots) and discusses their use and potential misuse in personalized persuasion. However, it does not report an actual incident of harm or a concrete event where an AI system has caused injury, rights violations, or other harms. Instead, it focuses on the plausible risks and ethical challenges posed by these AI capabilities, emphasizing the potential for future harm. Therefore, this is best classified as an AI Hazard, as it plausibly could lead to AI Incidents involving manipulation, emotional harm, or privacy violations, but no specific harm has been documented in the article.[AI generated]
AI principles
Privacy & data governanceDemocracy & human autonomy

Industries
Media, social platforms, and marketing

Affected stakeholders
ConsumersGeneral public

Harm types
PsychologicalPublic interest

Severity
AI hazard

Business function:
Marketing and advertisement

AI system task:
Interaction support/chatbotsContent generation


Articles about this incident or hazard

Thumbnail Image

Is your AI chatbot manipulating you? Subtly reshaping your opinions?

2026-05-12
The Conversation
Why's our monitor labelling this an incident or hazard?
The article clearly involves AI systems (large language models and chatbots) and discusses their use and potential misuse in personalized persuasion. However, it does not report an actual incident of harm or a concrete event where an AI system has caused injury, rights violations, or other harms. Instead, it focuses on the plausible risks and ethical challenges posed by these AI capabilities, emphasizing the potential for future harm. Therefore, this is best classified as an AI Hazard, as it plausibly could lead to AI Incidents involving manipulation, emotional harm, or privacy violations, but no specific harm has been documented in the article.
Thumbnail Image

Is your AI chatbot manipulating you? Subtly reshaping your opinions?

2026-05-12
Yahoo
Why's our monitor labelling this an incident or hazard?
The article clearly involves AI systems (large language models and chatbots) used for personalized persuasion. However, it does not describe a specific event where harm has already occurred but rather warns about the plausible future harms these AI systems could cause, such as manipulation, misinformation, and emotional exploitation. Therefore, it fits the definition of an AI Hazard, as it describes circumstances where AI use could plausibly lead to significant harms but does not document a concrete incident of harm yet.
Thumbnail Image

Is your AI chatbot manipulating you? Subtly reshaping your opinions?

2026-05-13
Stuff
Why's our monitor labelling this an incident or hazard?
The article describes AI systems (large language models/chatbots) that engage in deliberate emotional manipulation with potential to cause significant harm to individuals and communities by influencing opinions, behaviors, and decisions in a covert and personalized way. While no actual harm event is reported, the detailed explanation of how these AI systems could plausibly lead to harms such as misinformation, emotional exploitation, and manipulation of vulnerable users fits the definition of an AI Hazard. The focus is on the credible risk and potential for harm rather than a realized incident, so it is not an AI Incident. It is also not merely complementary information or unrelated, as the core narrative centers on the plausible harms from AI chatbot manipulation.
Thumbnail Image

Is your AI chatbot manipulating you?

2026-05-15
Knowridge Science Report
Why's our monitor labelling this an incident or hazard?
The article clearly involves AI systems (large language models and chatbots) used in personalized persuasion. It outlines how these systems could plausibly lead to significant harms including psychological harm, manipulation, and misinformation, which fall under harm to individuals and communities. However, it does not describe a specific incident where such harm has already occurred or been directly caused by an AI system. Instead, it presents a broad analysis and warning about the potential dangers and ethical concerns of AI-driven manipulation. This fits the definition of an AI Hazard, as it plausibly could lead to an AI Incident but does not report a concrete incident itself.
Thumbnail Image

Is your AI chatbot manipulating you? Subtly reshaping your opinions?

2026-05-13
NZCity
Why's our monitor labelling this an incident or hazard?
The article does not report a concrete event where harm has already occurred due to an AI system's use or malfunction, so it does not qualify as an AI Incident. Instead, it presents a reasoned analysis and warning about how AI chatbots could plausibly lead to significant harms such as manipulation, emotional exploitation, and erosion of independent judgment. This fits the definition of an AI Hazard, as it describes credible potential harms stemming from the development and use of AI systems for personalized persuasion and manipulation. The article also does not focus on responses, updates, or governance measures, so it is not Complementary Information. It is clearly related to AI systems and their societal impact, so it is not Unrelated.