AI Algorithms Linked to Surge in Online Crimes Against Minors in São Paulo

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

In São Paulo, AI-driven recommendation algorithms on digital platforms have contributed to a 78% rise in virtual crimes against children and adolescents during school holidays. These systems expose minors to harmful content, including hate speech and violent communities, resulting in increased victimization and psychological harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems explicitly through the recommendation algorithms that have directly contributed to increased exposure of minors to harmful and violent online content, leading to psychological and social harm. This fits the definition of an AI Incident because the AI system's use has directly led to harm to a group of people (minors) by facilitating access to extremist and violent content, which is a violation of their rights and harms their well-being. The article describes realized harm, not just potential risk, and the AI's role is pivotal in the chain of harm.[AI generated]
AI principles
SafetyRespect of human rights

Industries
Media, social platforms, and marketing

Affected stakeholders
Children

Harm types
Psychological

Business function:
Marketing and advertisement

AI system task:
Organisation/recommenders


Articles about this incident or hazard