Apple Removes AI Apps Generating Non-Consensual Nude Images from App Store

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Apple removed several AI-powered image generation apps from its App Store after reports revealed they created and promoted non-consensual explicit images, violating privacy and ethical standards. The apps were advertised on Instagram, claiming to undress women for free, prompting Apple to act following media exposure.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (generative AI for image creation) that have been used to produce non-consensual nude images, a form of sexual exploitation and violation of personal rights. The harm is realized as these images are generated and distributed, causing direct harm to individuals. Apple's removal of the apps is a response to this harm. The AI systems' development and use have directly led to violations of rights and harm to communities, fitting the definition of an AI Incident. The event is not merely a potential risk or complementary information but a concrete case of harm caused by AI misuse.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsSafetyHuman wellbeingAccountabilityFairnessTransparency & explainabilityRobustness & digital security

Industries
Media, social platforms, and marketingConsumer servicesIT infrastructure and hosting

Affected stakeholders
Women

Harm types
Human or fundamental rightsPsychologicalReputational

Severity
AI incident

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

"تولّد صوراً عارية".. أبل تزيل تطبيقات ذكاء اصطناعي من متجرها

2024-04-27
قناة العربية
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (generative AI for image creation) that have been used to produce non-consensual nude images, a form of sexual exploitation and violation of personal rights. The harm is realized as these images are generated and distributed, causing direct harm to individuals. Apple's removal of the apps is a response to this harm. The AI systems' development and use have directly led to violations of rights and harm to communities, fitting the definition of an AI Incident. The event is not merely a potential risk or complementary information but a concrete case of harm caused by AI misuse.
Thumbnail Image

لترويجها صورًا عارية بدون موافقة.. "أبل" تُزيل تطبيقات ذكاء اصطناعي من متجرها | المصري اليوم

2024-04-27
AL Masry Al Youm
Why's our monitor labelling this an incident or hazard?
The event involves AI systems used to generate explicit images without consent, which is a violation of personal rights and can cause harm to individuals. The misuse of AI in this way directly leads to harm (violation of rights and potential psychological harm). Apple's removal of these apps is a response to this harm. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to harm through the creation and distribution of non-consensual explicit images.
Thumbnail Image

تُصمم صورا غير لائقة.. "أبل" تزيل 3 تطبيقات للذكاء الاصطناعي

2024-04-27
الوطن
Why's our monitor labelling this an incident or hazard?
The event explicitly involves AI systems used for image generation. The harm arises from the AI-generated inappropriate images, which can be considered harm to communities and a violation of content standards. The AI systems' use directly led to this harm, prompting Apple to remove the apps. This fits the definition of an AI Incident, as the AI system's use has directly led to harm. The event is not merely a potential hazard or complementary information but a realized harm leading to action.
Thumbnail Image

أبل تزيل تطبيقات ذكاء اصطناعي من متجرها.. إليك السبب

2024-04-27
العين الإخبارية
Why's our monitor labelling this an incident or hazard?
The event involves AI systems used to generate images, specifically AI-powered apps that create nude images without consent. This misuse of AI directly leads to harm by violating individuals' rights and privacy, constituting a breach of fundamental rights. The removal of these apps follows reports of their harmful use, indicating that harm has occurred or is ongoing. Therefore, this qualifies as an AI Incident due to the direct harm caused by the AI system's use.
Thumbnail Image

آبل تزيل بعض تطبيقات الذكاء الاصطناعي من متجر آب ستور

2024-04-27
صحيفة المواطن الإلكترونية
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (AI-powered image generation apps) whose use has directly led to harm in the form of generating explicit images without consent, violating privacy and ethical standards. This constitutes a violation of human rights and ethical norms, fitting the definition of an AI Incident. The removal of these apps is a response to the realized harm caused by their misuse.
Thumbnail Image

حذف هوش مصنوعی غیراخلاقی از اپ استور

2024-04-28
خبرگزاری مهر | اخبار ایران و جهان | Mehr News Agency
Why's our monitor labelling this an incident or hazard?
The applications used generative AI to produce harmful content (non-consensual pornography), which is a clear violation of privacy and human rights. The AI system's use directly caused harm by enabling the creation and potential dissemination of such content. Therefore, this qualifies as an AI Incident. The article focuses on the harm caused and the response (removal) rather than just a general update or potential risk, so it is not merely Complementary Information or an AI Hazard.
Thumbnail Image

कार्रवाई : एपल ने लिया बड़ा एक्शन, App Store से रिमूव किए न्यूड फोटो बनाने वाले AI ऐप्स - Lalluram

2024-04-30
लल्लूराम
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (AI image generation apps) that were used to create non-consensual nude images, directly leading to harm in the form of violations of personal rights and potential psychological harm to individuals. The AI system's use facilitated the creation and dissemination of harmful content, fulfilling the criteria for an AI Incident. The article reports on the harm caused and the subsequent removal of these apps, indicating that the harm has occurred and is being addressed.
Thumbnail Image

AI ऐप से बना रहे थे न्यूड फोटो, Apple ने कर दिया बैन

2024-04-28
Navbharat Times
Why's our monitor labelling this an incident or hazard?
The apps explicitly used AI systems to generate manipulated nude images without user consent, causing violations of privacy and promoting obscenity, which are harms to individuals and communities. The AI system's misuse directly caused these harms, fulfilling the criteria for an AI Incident. Apple's removal of these apps is a response to this realized harm, not merely a precautionary measure, confirming the incident classification.
Thumbnail Image

Apple की बड़ी कार्रवाई, App Store से रिमूव किए न्यूड फोटो बनाने वाले AI ऐप्स

2024-04-30
आज तक
Why's our monitor labelling this an incident or hazard?
The AI system involved is the generative AI image generation feature within these apps, which was used to create harmful content (non-consensual nude images). This constitutes a violation of human rights and personal privacy, a clear harm caused by the AI system's use. Since the harm has occurred and the apps were actively enabling this misuse, this qualifies as an AI Incident. The article focuses on the harm caused and the response to it, not just potential future harm or general AI news.