EU Proposes Mandatory AI Scanning to Combat Online Child Abuse

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The EU Commission has proposed legislation requiring tech companies like Google and Facebook to use AI-powered software to scan for child sexual abuse material and grooming online. The plan includes establishing an EU center for technology support, but raises concerns about privacy and potential overreach.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article discusses a draft EU regulation that mandates the use of AI or automated systems to detect and remove child sexual abuse content online. This involves AI system use in the future to prevent harm. However, since the rules are still in proposal stage and no actual incident or harm has occurred, this qualifies as an AI Hazard due to the plausible future impact of AI systems in this context. The concerns about privacy and encryption also highlight potential risks associated with AI deployment under these rules.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsTransparency & explainabilityAccountabilityRobustness & digital securityDemocracy & human autonomy

Industries
Media, social platforms, and marketingDigital securityGovernment, security, and defence

Affected stakeholders
ChildrenGeneral public

Harm types
Human or fundamental rightsReputationalPublic interestPsychological

Business function:
Monitoring and quality controlCompliance and justice

AI system task:
Recognition/object detectionEvent/anomaly detection


Articles about this incident or hazard