LiblibAI Generates Inappropriate Content Due to Moderation Failure

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

LiblibAI, an AI content generation platform operated by Beijing Singularity Xingyu Technology, produced sexually explicit videos after users bypassed moderation with complex prompts. The incident, exposed by CCTV, highlighted flaws in content safety mechanisms. The company apologized, initiated technical fixes, and upgraded moderation to prevent future harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system generated inappropriate content that bypassed safety controls, directly leading to harm in the form of unsafe and non-compliant content dissemination. The company's response and remediation efforts are complementary information but do not negate the fact that the AI system's malfunction caused harm. Therefore, this qualifies as an AI Incident due to realized harm from the AI system's outputs.[AI generated]
AI principles
SafetyAccountability

Industries
Media, social platforms, and marketing

Affected stakeholders
BusinessGeneral public

Harm types
ReputationalPsychological

Business function:
Other

AI system task:
Content generation


Articles about this incident or hazard