OpenAI and Google AI Image Generators Bypassed to Produce Harmful Explicit Content

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI safety firm White Circle found that image generators from OpenAI and Google, including DALL-E 3 and Nano Banana 2, can be tricked into producing prohibited explicit content in about 13% of tests. The vulnerability enables the creation and spread of harmful non-consensual and illegal imagery, highlighting serious safety failures.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (text-to-image models) whose malfunction or inadequate safety guardrails have directly led to the generation and spread of harmful explicit content, including non-consensual intimate images and CSAM. This constitutes a violation of human rights and harm to communities, fulfilling the criteria for an AI Incident. The article documents realized harm, not just potential harm, and discusses the direct role of AI system vulnerabilities in enabling this harm. Therefore, the event is classified as an AI Incident.[AI generated]
AI principles
SafetyRobustness & digital security
Industries
Media, social platforms, and marketing
Affected stakeholders
General public
Harm types
Human or fundamental rightsPsychologicalReputational
Business function:
Other
AI system task:
Content generation

Articles about this incident or hazard