OpenAI's ChatGPT Human Review Raises Privacy and Harm Concerns

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

OpenAI employs hundreds of contractors to review real ChatGPT user conversations for model improvement, sometimes exposing sensitive personal data despite privacy filters. This practice, largely unknown to users, has led to privacy concerns, lawsuits alleging harm, and questions about the adequacy of data protection measures.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an AI system (ChatGPT) whose user conversations are used for training and improvement, with human reviewers accessing real user data, including sensitive information. The imperfect privacy filtering means personal data can be exposed to humans, constituting a violation of privacy rights and potentially other legal protections. This is a direct harm linked to the AI system's use and data processing practices. The article describes actual ongoing practices leading to these harms, not just potential risks, so it qualifies as an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
Privacy & data governanceTransparency & explainability

Industries
IT infrastructure and hosting

Affected stakeholders
Consumers

Harm types
Human or fundamental rights

Business function:
Research and development

AI system task:
Interaction support/chatbotsContent generation


Articles about this incident or hazard