AI Algorithms in China Perpetuate Gender Bias and Discrimination

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A report by the Mana Data Foundation and UN Women found that Chinese AI systems, including search engines and recruitment algorithms, systematically perpetuate gender bias. These systems return male-dominated results for professional roles and derogatory content for female-related searches, and discriminate against women in job applications, causing real-world harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (search engine and recommendation algorithms) whose use has directly led to realized harms, specifically gender discrimination and the perpetuation of harmful cultural stereotypes. These harms constitute violations of human rights and harm to communities as defined in the framework. The article provides concrete examples of biased outputs and offensive content resulting from these AI systems, fulfilling the criteria for an AI Incident. The discussion of mitigation efforts and developer awareness does not overshadow the fact that harm is occurring due to the AI systems' outputs.[AI generated]
AI principles
FairnessRespect of human rightsAccountabilityTransparency & explainability

Industries
Media, social platforms, and marketingBusiness processes and support services

Affected stakeholders
Women

Harm types
Human or fundamental rightsEconomic/PropertyPsychological

Business function:
Human resource management

AI system task:
Organisation/recommendersGoal-driven organisation


Articles about this incident or hazard