Multiple Harms from ChatGPT Use: Legal, Health, and Privacy Risks

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Several incidents involving ChatGPT have led to real-world harms, including fabricated legal citations causing court penalties, privacy breaches from sharing sensitive data, mental and physical health risks due to misleading advice, and exposure of confidential information. These cases highlight significant risks when relying on AI systems for sensitive or critical tasks.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly describes multiple instances where ChatGPT's use has directly or indirectly caused harm to individuals (psychological harm, legal harm), organizations (data breaches, security risks), and legal systems (fabricated legal citations). The harms are realized and documented, not hypothetical. The AI system's involvement is clear and central to these harms. Therefore, this event qualifies as an AI Incident under the OECD framework.[AI generated]
AI principles
Privacy & data governanceRobustness & digital security

Industries
Government, security, and defenceHealthcare, drugs, and biotechnology

Affected stakeholders
Consumers

Harm types
Economic/PropertyHuman or fundamental rightsPsychological

AI system task:
Interaction support/chatbotsContent generation


Articles about this incident or hazard