Google AI Push Notification Includes Racial Slur, Prompts Apology

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Google issued an AI-generated push notification referencing a BAFTA Film Awards incident, but the alert included a racial slur. The notification caused public outrage and harm by spreading offensive language. Google apologized, removed the alert, and pledged to improve safeguards to prevent similar AI-generated content errors.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system was involved in generating the news alert that included offensive content, which directly caused harm to communities by spreading harmful language and causing public outrage. This fits the definition of an AI Incident because the AI system's use directly led to harm (harm to communities through offensive content dissemination). The event is not merely a potential hazard or complementary information but a realized harm caused by AI system malfunction or misuse.[AI generated]
AI principles
FairnessSafety

Industries
Media, social platforms, and marketing

Affected stakeholders
BusinessGeneral public

Harm types
PsychologicalReputationalHuman or fundamental rights

Severity
AI incident

Business function:
Other

AI system task:
Content generation


Articles about this incident or hazard