Chinese Platforms Crack Down on AI-Driven Elderly Scams and Misinformation

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Douyin and Baidu's Baijiahao have intensified efforts to combat AI-driven harms. Douyin used AI moderation to ban accounts exploiting elderly users through impersonation, emotional manipulation, and scams. Baijiahao removed over 15,000 AI-generated false news items and banned 489 accounts spreading misinformation, highlighting the risks of AI-generated content to vulnerable groups and public trust.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (AIGC) used to generate false and misleading content, which constitutes harm to communities by spreading misinformation and false news. The AI system's use has directly led to this harm, fulfilling the criteria for an AI Incident. The platform's enforcement actions are responses to this harm, not the primary event. Therefore, this is classified as an AI Incident due to realized harm caused by AI-generated misinformation.[AI generated]
AI principles
AccountabilitySafetyTransparency & explainabilityRobustness & digital securityHuman wellbeingDemocracy & human autonomyPrivacy & data governance

Industries
Media, social platforms, and marketingDigital security

Affected stakeholders
OtherGeneral public

Harm types
Economic/PropertyPsychologicalPublic interestReputational

Severity
AI incident

AI system task:
Content generationInteraction support/chatbots


Articles about this incident or hazard

Thumbnail Image

百度百家号打击自媒体利用AI传播违规内容 封禁违规账号489个

2023-12-13
chinaz.com
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (AIGC) used to generate false and misleading content, which constitutes harm to communities by spreading misinformation and false news. The AI system's use has directly led to this harm, fulfilling the criteria for an AI Incident. The platform's enforcement actions are responses to this harm, not the primary event. Therefore, this is classified as an AI Incident due to realized harm caused by AI-generated misinformation.
Thumbnail Image

保障老年人健康触网,抖音严打冒充名人、土味情话诱导等行为

2023-12-15
新民网 - 为民分忧 与民同乐
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems implicitly, as TikTok's content moderation and detection of impersonation, face-swapping, and misleading content likely rely on AI technologies for automated detection and enforcement. The harms addressed include exploitation and deception of elderly users, which constitute harm to communities and individuals (harm category d). However, the article focuses on the platform's governance and mitigation efforts rather than describing a specific incident where AI caused harm. Therefore, this is best classified as Complementary Information about societal and governance responses to AI-related harms in the online content ecosystem.
Thumbnail Image

抖音升级"老年人权益保护专项"治理 封禁数个违规百万粉丝账号

2023-12-14
chinanews.com.cn
Why's our monitor labelling this an incident or hazard?
The event involves AI systems used by Douyin to detect and moderate content and accounts that exploit elderly users, leading to realized harm through deception and exploitation. The platform's AI-driven moderation directly addresses violations that harm a vulnerable group, fulfilling the criteria for an AI Incident. The harms are not just potential but have been occurring, as evidenced by the large number of accounts and content removed or banned. This is not merely complementary information or a hazard, but a clear incident involving AI system use leading to harm.