AI Algorithms Used for Content Suppression and Low-Quality Content Cleanup in China

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Chinese authorities removed nearly 40,000 accounts and 708,000 posts to combat AI-generated low-quality content, misinformation, and illegal activity. Simultaneously, coordinated manipulation of platform algorithms by fake accounts suppressed dissenting voices, causing psychological harm and limiting freedom of expression. Both incidents highlight AI's role in online harm and censorship.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use and manipulation of AI systems—specifically, the recommendation and moderation algorithms of a social media platform—to indirectly cause harm by silencing dissenting voices. The harm includes violation of rights (freedom of expression) and harm to communities (suppression of discourse). The AI system's involvement is direct and pivotal, as the attack exploits algorithmic vulnerabilities to cause the harm. Hence, it meets the criteria for an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
Respect of human rightsDemocracy & human autonomy
Industries
Media, social platforms, and marketing
Affected stakeholders
General publicCivil society
Harm types
PsychologicalHuman or fundamental rights
Severity
AI incident
AI system task:
Event/anomaly detectionOrganisation/recommenders

Articles about this incident or hazard