Unauthorized Use of Brazilian Children's Photos for AI Training

The Human Rights Watch report documents that 170 personal images of children were included in LAION-5B without parental consent. This data sourcing for AI training has already occurred and directly breaches the children’s privacy and data protection rights, meeting the criteria for a realized AI Incident.

The incident involves the development of an AI system (training on LAION-5B) using personal data without consent, directly violating children’s privacy rights (a human rights breach). This is a realized harm from the AI system’s use of unauthorized images, fitting the definition of an AI Incident.

The article describes how an AI-related dataset (LAION-5B) was found to contain personal images of minors without their consent—an unauthorized use of private data that breaches fundamental rights. This misuse directly implicates the AI system’s development (training on unconsented images) and poses real harm, qualifying it as an AI Incident under violations of human rights/privacy.

The LAION-5B dataset (used to train AI systems) directly led to human rights violations—unauthorized use of children’s images, privacy harms, and facilitation of sexual exploitation via AI. These harms have materialized and AI systems play a pivotal role in both the collection and misuse of the data, constituting an AI Incident.

The article reports actual misuse of an AI training dataset (LAION-5B) containing identifiable images of children, resulting in violations of privacy and facilitating sexual exploitation through AI-generated content. This constitutes a direct harm to fundamental rights and falls under an AI incident.

The article describes an actual misuse of AI tools—datasets assembled without consent and deepfake creation—that directly infringes on children’s rights and places them at risk, fitting the definition of an AI Incident (violations of fundamental rights).

This event describes actual harms resulting from the development and misuse of AI systems: unauthorized collection of children’s images for AI training (privacy breach and rights violation) and the generation of sexual deepfakes used to harass minors (psychological and sexual harm). These are direct violations of human rights and node (c) and (a) in the harm categories, making it an AI Incident.

An AI system (large vision models trained on the LAION-5B dataset) has been developed and used in a way that directly leads to privacy violations and potential sexual exploitation of children. The misuse of children’s images without consent for AI training and the creation of harmful deepfakes inflicts real harm to minors’ rights and safety. Thus, it meets the criteria for an AI Incident.

Human Rights Watch found 170 personal images of minors—depicting moments like births and school events—scraped from websites and social media without consent and used to train AI models. This direct misuse of personal data infringes on individuals’ rights, qualifying as an AI Incident under violations of human rights and applicable privacy laws.

The unauthorized use of minors’ images for AI training is a breach of fundamental human rights (privacy and consent). It directly involves the development of an AI system (the LAION-5B dataset) leading to a tangible rights violation. This meets the criteria for an AI Incident under violations of human rights.

The event involves development and use of an AI system (image dataset for training generative models) that directly led to violations of children’s rights, privacy breaches, and documented cases of sexual harassment via deepfake images. These constitute realized harms (privacy and human rights violations, sexual harassment), so this is an AI Incident.

Images of minors were scraped without consent and incorporated into an AI training dataset (LAION-5B). Although no direct incident of harm (such as a leak or deepfake generation) has been reported, the unauthorized use of sensitive personal data for AI training poses plausible future risks of privacy violations and malicious deepfake creation. Therefore, this situation is classified as an AI Hazard.

Photos of identifiable minors were used without authorization to train AI models (LAION-5B), constituting a direct violation of their privacy and rights. The use of these images for malicious deepfakes has already created real risks of exploitation and harm. This meets the criteria for an AI Incident, as the development and use of the AI system directly led to harm.

The incident involves an AI‐related dataset (LAION‐5B) that has already been used to train models for startups. It directly led to a violation of privacy rights (children’s personal data and images were scraped and stored without consent) and poses further risks (potential full reproduction by models and generation of pornographic imagery). This is a concrete harm linked to an AI system’s training data, classifying it as an AI Incident.

Personal images of minors were scraped and used without consent to train AI models, leading to privacy breaches and reported deepfake harassment of at least 85 girls. This is a direct violation of personal rights through the development and use of an AI system, qualifying as an AI Incident.

An AI system (LAION-trained generative models and resulting deepfake tools) has directly led to harms: generation of child sexual abuse material and use of deepfakes by criminals to defraud victims. These constitute violations of children’s rights and privacy, as well as real-world fraud and social harm, meeting the definition of an AI Incident.

The event involves an AI system’s use of non-consensual personal data (children’s photos) in model training, directly violating privacy and children’s rights (human rights breach). The unauthorized scraping and inclusion of sensitive information in a public dataset already constitutes realized harm linked to AI development and use.

The scraping and inclusion of minors’ personal photos in an AI training corpus without consent constitutes a violation of fundamental rights and has directly contributed to potential harmful deepfake generation. This misuse of personal data for AI development is a realized harm to privacy and human rights.

The event describes the actual unauthorized collection and use of minors’ personal data and images for AI training, directly breaching privacy rights and YouTube’s terms of service. It has already led to child sexual abuse material risks and privacy violations, meeting the definition of an AI Incident under human rights violations.

Human Rights Watch uncovered that images of minors were integrated into LAION-5B without authorization, meaning AI systems have been trained using personal data in breach of privacy laws and children’s rights. This constitutes a realized harm—violation of human rights/privacy—and paves the way for malicious uses (e.g., deepfakes).

This is a direct misuse of an AI training dataset leading to violations of children’s privacy and personal data rights. The development and use of LAION-5B have already resulted in identifiable harm to individuals, meeting the criteria for an AI Incident under rights violations.

This is a realized harm where AI development processes (training datasets for AI systems) are directly using images of minors without consent, violating fundamental privacy and human rights. Therefore, it constitutes an AI Incident.

The report shows that AI systems (LAION-5B–trained models and associated deepfake tools) have already been trained on and are using images of minors without consent, resulting in actual rights violations and significant privacy harms. This is more than a hypothetical risk: personal data have been scraped and misused, meeting the criteria for an AI Incident (violation of human rights and privacy obligations).

Images of minors were harvested without consent to train AI algorithms, and are already being misused to create deepfakes, posing real privacy and safety harms. This is a realized incident of human-rights violations caused by AI systems.

The event involves an AI system (the LAION-5B dataset used to train AI models) whose development included unauthorized use of personal images of children and adolescents, constituting a violation of rights under applicable law protecting privacy and intellectual property. The AI system's role is pivotal as it enables the creation and dissemination of non-consensual deepfakes, which can cause harm to individuals and communities. Therefore, this qualifies as an AI Incident due to realized violations of rights and potential harms stemming from the AI system's development and use.

The event involves AI systems trained on large datasets containing personal images of children without consent, leading to direct violations of privacy and potential misuse causing harm to children. The use of these AI systems has already resulted in harm through privacy breaches and the generation of explicit images, which is a clear violation of rights and a significant harm. Therefore, this qualifies as an AI Incident due to realized harm linked to AI system development and use.

The article explicitly involves an AI system (AI tools trained on the LAION-5B dataset) whose development and use have directly caused harm by violating children's privacy rights and enabling harmful manipulations (deepfakes) that have led to sexual harassment. The harms include violations of fundamental rights and harm to individuals and communities. The AI system's role is pivotal as the dataset and AI tools trained on it are the basis for these harms. Hence, this event meets the criteria for an AI Incident rather than a hazard or complementary information.

The event explicitly involves an AI system (an AI training platform) using images without consent, which constitutes a violation of privacy rights (a breach of obligations under applicable law protecting fundamental rights). The use of these images for AI training has already occurred, causing realized harm (privacy invasion). Additionally, the potential for manipulative or criminal use of these images represents plausible future harm. Therefore, this qualifies as an AI Incident due to the direct and indirect harms caused by the AI system's development and use.

The article explicitly states that photos of children were used without consent in an AI training dataset, which is a direct violation of privacy and potentially other rights. The AI system's development involved using these images, leading to harm in terms of rights violations. The harm is realized as the images were already incorporated into the dataset. Hence, this is an AI Incident rather than a hazard or complementary information.

The event involves an AI system trained on a dataset containing unauthorized images of children, which is a direct use of AI development practices. The resulting AI tools can be used to create harmful deepfakes, which have already placed children at risk of exploitation and harm. This constitutes a violation of rights and harm to individuals, fulfilling the criteria for an AI Incident. The involvement of AI in both the dataset creation and the generation of malicious content is explicit and directly linked to realized harm and rights violations.

The report shows that an AI system’s development (LAION-5B dataset) directly led to privacy violations and the misuse of children’s images for deepfakes and sexual harassment, constituting a violation of human rights.

The event describes the misuse and harm of AI systems: models trained on illegally obtained images of children have generated explicit sexual content and deepfake child pornography, directly harming minors and violating Brazilian data protection and child exploitation laws. These are realized harms caused by AI development and use.

AI systems (image generators trained on Laion-5B) directly produced sexually explicit and violent depictions of real children without consent. The training data was illegally scraped, enabling AI models to create realistic child pornography, which has been used to harass and traumatize minors. Actual harms (privacy violations, child sexual exploitation, harassment) have occurred, meeting the criteria for an AI Incident.

The event describes actual harms arising from the development and use of an AI system. The LAION-5B dataset’s unlawful ingestion of children’s personal data and subsequent generation of explicit imagery directly breaches legal protections for minors and has produced sexual exploitation and harassment via deepfake images. This constitutes a clear AI Incident involving violation of human rights (privacy, protection of minors) and harm to individuals.

The event describes an AI system’s training process directly using private data of minors without their knowledge or consent, constituting a violation of personal and children’s rights. This misuse of data has already enabled harmful content generation, fulfilling the criteria for an AI Incident under human rights violations.

The scraping and inclusion of children’s photos and personal data in an AI training dataset has already occurred and directly breaches their privacy and rights. The misuse of this data by AI image generators, including creating explicit or harmful deepfakes, demonstrates realized harm and a direct link to an AI system’s training data. Therefore, this event is an AI Incident.

Children’s images were scraped without knowledge or consent (data misuse in model development), AI models trained on that data reproduced or manipulated children’s likenesses, and at least 85 girls experienced harassment when deepfakes of them were circulated. This constitutes direct, realized harm to privacy and human rights, meeting the criteria for an AI Incident.

An AI system (generative image models trained on LAION-5B) was directly involved: children’s images were used without consent to train the models, and those models were then used to produce harmful deepfakes. This misuse has led to actual harm—privacy breaches, sexual harassment, and exploitation—constituting an AI Incident under the rights violation and harm criteria.

The event describes actual scraping of children’s photos and personal data into an open-source AI training dataset, a misuse of data that violates fundamental privacy and exposes children to potential harm. These harms (privacy violations, potential creation of CSAM and manipulative deepfakes) are direct consequences of the AI system’s use of the data, fitting the definition of an AI Incident (rights violations under category c).

An AI system (LAION-5B dataset powering image generators such as Stable Diffusion) was built using children’s personal photos scraped from the web without consent. This has directly led to violations of privacy and enabled the creation of non-consensual, sexually explicit deepfakes of minors—harm to their rights and well-being. These realized harms from AI-generated content classify this as an AI Incident.

The event involves an AI system trained on a dataset containing children's photos without consent, which is a violation of privacy and fundamental rights. The report documents actual use of these images in AI training and instances of harmful deepfake creation, indicating realized harm. Therefore, this qualifies as an AI Incident due to the direct link between AI system development/use and violations of rights and potential exploitation of children.

The event involves AI systems (image generators like Stable Diffusion) trained on datasets containing children's images without consent, which is a direct violation of privacy and human rights. The harms are realized, including bullying and creation of sexually explicit deepfake content targeting minors, which are clear violations of rights and harm to individuals and communities. The AI system's development and use are central to these harms, fulfilling the criteria for an AI Incident rather than a hazard or complementary information.

The event explicitly involves an AI system (text-to-image models like Stable Diffusion) trained on a dataset containing personal images of children scraped without consent. The use of these images infringes on privacy rights and human rights, constituting harm (violation of rights). The AI system's outputs have been used to create synthetic CSAM, a serious harm. The presence of these images in the training data directly contributes to these harms. The event also highlights ongoing risks and calls for regulatory responses. Given the realized harms and the AI system's pivotal role, this is classified as an AI Incident.

The event involves an AI system (LAION-5B dataset and AI models trained on it) whose development and use have directly led to harm: children's personal images were used without consent, enabling the creation of malicious deepfakes that have caused harassment and exploitation. This is a clear violation of privacy and children's rights, fulfilling the criteria for an AI Incident. The involvement of AI in generating harmful content and the direct impact on children's safety and rights confirm this classification.

The event involves an AI system trained on a dataset containing personal images of children without consent, leading to the creation of deepfake images that can be exploited for harm. This misuse of data and the resulting potential for exploitation and harm to children fits the definition of an AI Incident, as it directly leads to violations of rights and harm to individuals. The involvement of the AI system in generating harmful content and the direct link to harm justify classification as an AI Incident.

The article describes an AI system being developed using scraped personal photos of children without consent, which constitutes a violation of privacy and potentially intellectual property rights. The AI tools trained on this data are then used to create malicious deepfakes, which can cause harm to the children by misrepresenting them and exposing them to risks. This direct misuse and resulting harm fit the definition of an AI Incident, as it involves violations of rights and harm to individuals caused by the development and use of AI systems.

The report explicitly states that AI tools were trained on photos of children retrieved without consent, which constitutes a violation of privacy and intellectual property rights. Furthermore, the AI-generated deepfakes have caused real harm to children, including harassment and sexual exploitation, fulfilling the criteria for an AI Incident. The involvement of AI in generating harmful content and the use of illegally obtained data directly link the AI system to realized harm, not just potential harm.

The event describes actual misuse of personal data—photos of children—without consent for AI training. This constitutes a violation of fundamental rights and a breach of obligations intended to protect personal data, directly meeting the criteria for an AI Incident under “violations of human rights or a breach of obligations under applicable law.”

The unconsented collection and use of children’s images for AI training directly infringes on fundamental privacy and child protection rights (a breach of human rights). The AI system’s development and use have already led to these violations and pose further risks of manipulation and abuse. Therefore, this is an AI Incident.

This is an AI Incident because the development of generative AI systems directly involved the use of unconsented images of minors, leading to violations of children’s rights (privacy, protection from exploitation) and creating real-world harms. The dataset’s misuse in training models constitutes a direct link between the AI system’s development and human-rights violations.

The event describes AI systems being trained illegally on private images of minors—directly violating their rights and leading to real-world harms (child deepfakes and harassment). This is a realized violation of human rights under the framework, so it qualifies as an AI Incident.

An AI system (generative models trained on LAION-5B) has been used without consent to produce harmful deepfake content of minors, causing real harassment and privacy harms. This constitutes a realized violation of children’s rights and personal safety facilitated by AI, fitting the definition of an AI Incident.

The event explicitly involves an AI system (generative image models trained on LAION-5B) and the use of data without consent, which is a breach of fundamental rights (privacy and protection of children). The harm is realized as the AI models are trained on these images, enabling potential misuse and privacy violations. Therefore, this qualifies as an AI Incident due to violations of human rights and harm to individuals caused by the AI system's development and use.

Human Rights Watch and Stanford found that LAION-5B scraped thousands of personal photos of children without parental consent—violating Brazilian law—and that image generators (e.g., Stable Diffusion variants, Civiai) trained on this data have produced sexualized and violent images of minors. Deepfakes have been used to harass girls at school. These actions violate children’s rights and have directly caused psychological and privacy harms.

The event describes the unauthorized extraction and use of minors’ personal images in an AI training dataset (LAION-5B) for Stable Diffusion, constituting a breach of fundamental rights (privacy) directly caused by the AI system’s development and use. Therefore, it qualifies as an AI Incident.

This event describes the actual misuse of an AI system—deepfake generators trained on LAION-5B—which has directly led to harm: child privacy violations, sexual exploitation, and harassment. Because the AI’s training and subsequent outputs have caused real-world harm to children’s rights and safety, it meets the criteria for an AI Incident.

The event describes an actual privacy violation—AI training datasets including minors’ images without consent—constituting a breach of children’s rights and exposing them to concrete harms (deepfakes, abuse). The misuse occurred during the development and deployment of AI systems (image generators), directly leading to rights violations, fitting the definition of an AI Incident.

The event explicitly involves an AI system (generative AI models trained on LAION-5B dataset) and the unauthorized use of personal data of children without consent. This use directly breaches privacy rights and applicable laws protecting personal data, which fits the definition of harm under violations of human rights or breach of legal obligations. The harm is realized, not just potential, as the images were already used in training. Hence, this is an AI Incident rather than a hazard or complementary information.