AI Training Dataset Violates Children's Privacy

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Human Rights Watch reports that photos of Australian children, including indigenous and identifiable ones, have been used without consent in the LAION-5B AI training dataset. This raises privacy concerns and potential legal breaches, prompting calls for urgent legal reforms to protect children's rights and prevent misuse of AI tools.[AI generated]

Why's our monitor labelling this an incident or hazard?

Researchers from Human Rights Watch found that the LAION-5B dataset, used to train widely deployed AI models (e.g., Stable Diffusion, Midjourney), scraped private and semi-private images of minors without consent, including detailed personal metadata. This data misuse is already realized, breaches fundamental privacy and human rights, and poses actual harms (e.g., potential for hyper-realistic deepfakes of children). Thus it meets the criteria for an AI Incident.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsTransparency & explainabilityAccountabilityFairnessHuman wellbeing
Industries
Media, social platforms, and marketingGovernment, security, and defence
Affected stakeholders
Children
Harm types
Human or fundamental rights
Severity
AI incident
Business function:
Research and development
AI system task:
Content generationRecognition/object detection

Articles about this incident or hazard