AI Algorithms in China Perpetuate Gender Bias and Discrimination

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A report by the Mana Data Foundation and UN Women found that Chinese AI systems, including search engines and recruitment algorithms, systematically perpetuate gender bias. These systems return male-dominated results for professional roles and derogatory content for female-related searches, and discriminate against women in job applications, causing real-world harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems (search engine and recommendation algorithms) whose use has directly led to realized harms, specifically gender discrimination and the perpetuation of harmful cultural stereotypes. These harms constitute violations of human rights and harm to communities as defined in the framework. The article provides concrete examples of biased outputs and offensive content resulting from these AI systems, fulfilling the criteria for an AI Incident. The discussion of mitigation efforts and developer awareness does not overshadow the fact that harm is occurring due to the AI systems' outputs.[AI generated]
AI principles
FairnessRespect of human rightsAccountabilityTransparency & explainability

Industries
Media, social platforms, and marketingBusiness processes and support services

Affected stakeholders
Women

Harm types
Human or fundamental rightsEconomic/PropertyPsychological

Severity
AI incident

Business function:
Human resource management

AI system task:
Organisation/recommendersGoal-driven organisation


Articles about this incident or hazard

Thumbnail Image

How Chinese artificial intelligence perpetuates gender biases

2021-09-30
South China Morning Post
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (search engine and recommendation algorithms) whose use has directly led to realized harms, specifically gender discrimination and the perpetuation of harmful cultural stereotypes. These harms constitute violations of human rights and harm to communities as defined in the framework. The article provides concrete examples of biased outputs and offensive content resulting from these AI systems, fulfilling the criteria for an AI Incident. The discussion of mitigation efforts and developer awareness does not overshadow the fact that harm is occurring due to the AI systems' outputs.
Thumbnail Image

The 'CEO' is a man: how Chinese artificial intelligence perpetuates gender biases

2021-09-30
The Star
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (search engine algorithms, recommendation algorithms) whose use has directly led to harms such as gender discrimination and dissemination of offensive content, which constitute violations of human rights and harm to communities. The biases are systemic and documented with concrete examples, fulfilling the criteria for an AI Incident. The article also mentions remediation efforts, but the primary focus is on the existence and impact of the biased AI systems, not just on responses, so it is not merely Complementary Information.
Thumbnail Image

AsiaOne

2021-10-01
AsiaOne
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly (search engines, advertising algorithms, deep learning models) whose outputs have directly caused harm by perpetuating gender discrimination and offensive content, violating human rights and harming communities. The article documents realized harms, not just potential risks, and includes examples of biased search results and advertisements. The involvement of AI in causing these harms is clear and direct, fulfilling the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Study finds gender bias present in artificial intelligence algorithms

2021-09-28
SHINE
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions AI systems exhibiting gender discrimination that negatively impacts women's job application acceptance rates and facial recognition accuracy. This constitutes a violation of rights and harm to communities due to biased AI outputs. Since these harms are occurring and the AI systems are directly involved, this qualifies as an AI Incident. The article focuses on the harm caused by AI bias rather than just potential risks or responses, so it is not merely Complementary Information or an AI Hazard.
Thumbnail Image

The 'CEO' is a man: how Chinese artificial intelligence perpetuates gender biases

2021-10-02
IntellAsia
Why's our monitor labelling this an incident or hazard?
The presence of AI systems is explicit, as the article focuses on AI algorithms used in search engines, advertising, and content moderation. The harms described include gender discrimination and offensive content dissemination, which are violations of human rights and cause harm to communities. These harms have already occurred, making this an AI Incident rather than a hazard. The article also discusses remediation efforts, but the primary focus is on the realized harms caused by AI biases.