AI Algorithms Promote and Recommend Neo-Nazi Products on Major Platforms

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Amazon, Google, and Wish removed neo-Nazi and white-supremacist products after a BBC investigation revealed their AI-powered recommendation algorithms were promoting and suggesting such items. The incident highlights the role of AI systems in amplifying harmful, extremist content before company intervention and removal.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event describes AI-powered recommendation algorithms on major platforms that were promoting hateful and extremist products, which is a violation of human rights and causes harm to communities. The harm is realized as these products were available and recommended, potentially facilitating hate and extremist activities. The companies' removal of the content and actions against sellers are responses to this harm. Since the AI systems' use directly contributed to the dissemination of harmful content, this qualifies as an AI Incident.[AI generated]
AI principles
AccountabilitySafetyRobustness & digital securityTransparency & explainabilityRespect of human rightsDemocracy & human autonomyHuman wellbeing

Industries
Logistics, wholesale, and retail

Affected stakeholders
General public

Harm types
Public interestPsychologicalHuman or fundamental rights

Business function:
Marketing and advertisement

AI system task:
Organisation/recommenders


Articles about this incident or hazard