Misleading AI-Generated Summaries Distort Human Memory

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A study by Georgetown University and the University of Washington found that AI-generated summaries, especially those with inaccuracies or omissions, can significantly distort human memory and perception of events. This effect persists even when users know the summaries are AI-generated, raising concerns for high-stakes contexts like police footage analysis.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly describes an AI system generating summaries that omit important information, leading to people recalling events inaccurately and misattributing fault. This is a direct harm to individuals' understanding and memory, which falls under harm to people and communities. The AI system's malfunction (producing misleading summaries) directly caused this harm. The study's context includes real-world applications such as police footage analysis, highlighting the significance of the harm. Hence, the event meets the criteria for an AI Incident.[AI generated]
AI principles
Robustness & digital securitySafety

Industries
Government, security, and defence

Affected stakeholders
Consumers

Harm types
Psychological

AI system task:
Content generation


Articles about this incident or hazard