ChatGPT Frequently Gives Incorrect Programming Answers, Study Finds

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A Purdue University study found that ChatGPT provides incorrect answers to programming questions over 50% of the time. Despite this, users often prefer its well-articulated responses, leading to potential harm through misinformation, coding errors, and reputational risks for developers who rely on its outputs.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use of an AI system (ChatGPT) whose outputs have directly led to misinformation and potential harm to users seeking software engineering advice. The harm is indirect but significant, as incorrect answers can lead to flawed software development, wasted time, or other negative consequences. Therefore, this qualifies as an AI Incident due to realized harm stemming from the AI system's use.[AI generated]
AI principles
AccountabilityRobustness & digital securitySafetyTransparency & explainabilityDemocracy & human autonomyHuman wellbeing

Industries
IT infrastructure and hostingEducation and trainingDigital security

Affected stakeholders
Workers

Harm types
ReputationalEconomic/Property

Business function:
Research and development

AI system task:
Interaction support/chatbotsContent generationReasoning with knowledge structures/planning


Articles about this incident or hazard