China's COVID-19 AI Surveillance Expands to Mass Social Control

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

China's government used AI-driven big data systems to monitor COVID-19, collecting personal and biometric data to assign health risk codes. These systems are now being repurposed for broader mass surveillance and social credit enforcement, resulting in privacy breaches, data hacks, and increased social control, raising significant human rights concerns.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an AI system (big data analytics and biometric verification integrated into a health code app) used for pandemic control and extended for mass surveillance. The system's outputs directly influence individuals' freedoms (e.g., quarantine enforcement, travel restrictions) and are linked to violations of human rights and privacy. The article documents realized harms including coercive control, social credit punishments, and data breaches exposing sensitive personal information. These constitute violations of fundamental rights and harm to communities. Hence, the event meets the criteria for an AI Incident as the AI system's use has directly led to significant harms.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsTransparency & explainabilityAccountabilityRobustness & digital securityDemocracy & human autonomy

Industries
Government, security, and defence

Affected stakeholders
General public

Harm types
Human or fundamental rightsPublic interest

Severity
AI incident

Business function:
Compliance and justice

AI system task:
Forecasting/prediction


Articles about this incident or hazard

Thumbnail Image

China's 'surveillance creep': How big data COVID monitoring could be used to control people post-pandemic

2021-08-31
Economic Times
Why's our monitor labelling this an incident or hazard?
The event involves AI systems in the form of big data analytics and automated health code risk assessments used for pandemic control and now extended to mass surveillance. Although no direct harm is reported as having occurred, the article outlines a credible and plausible risk that these AI systems could lead to violations of human rights and societal harm through function creep and expanded surveillance powers. The use of biometric data, health status, and movement tracking analyzed by AI to control and potentially coerce populations fits the definition of an AI Hazard, as it could plausibly lead to significant harm in the future. The article does not describe a realized incident but warns of the potential consequences, making AI Hazard the appropriate classification.
Thumbnail Image

'China's Surveillance Creep': How Big Data Covid Monitoring Could Be Used to Control People Post-pandemic

2021-08-31
News18
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (big data analytics and biometric verification integrated into a health code app) used for pandemic control and extended for mass surveillance. The system's outputs directly influence individuals' freedoms (e.g., quarantine enforcement, travel restrictions) and are linked to violations of human rights and privacy. The article documents realized harms including coercive control, social credit punishments, and data breaches exposing sensitive personal information. These constitute violations of fundamental rights and harm to communities. Hence, the event meets the criteria for an AI Incident as the AI system's use has directly led to significant harms.
Thumbnail Image

China's 'surveillance creep': how big data COVID monitoring could be used to control people post-pandemic

2021-08-31
The Conversation
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (big data analytics and risk assessment algorithms processing personal health and movement data) whose use has directly led to harms including privacy violations, potential human rights infringements, and social coercion. The system's malfunction or misuse (e.g., data hacking) and function creep to broader surveillance and social credit enforcement demonstrate direct harm to individuals and communities. The article documents realized harms and societal impacts, not just potential risks, thus classifying it as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

China's 'surveillance creep': how big data Covid-19 monitoring could be used to control people post-pandemic

2021-08-31
Stuff
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (big data collection, analysis, and health code risk assessment) in the development and use phases. The AI system's outputs have directly led to harms including violations of privacy, potential breaches of fundamental rights, and coercive social control measures (e.g., social credit system punishments). The article documents realized harms such as data hacking, misuse of biometric data, and function creep leading to expanded surveillance beyond the original health purpose. These constitute violations of human rights and harm to communities, meeting the criteria for an AI Incident rather than a hazard or complementary information. The AI system's role is pivotal in enabling these harms through automated risk assessments and data integration at scale.
Thumbnail Image

World News | China's 'surveillance Creep': How Big Data COVID Monitoring Could Be Used to Control People Post-pandemic | LatestLY

2021-08-31
LatestLY
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (big data analytics, biometric verification, risk scoring algorithms) in surveillance and social credit systems. The use of these AI systems has directly led to harms including privacy violations, potential coercion, and restrictions on freedoms, which constitute violations of human rights and harm to communities. The article documents realized harms (e.g., data breaches, social credit punishments) and ongoing use of AI surveillance beyond the pandemic, indicating an AI Incident rather than a mere hazard or complementary information. The AI system's role is pivotal in enabling these harms through automated data collection, risk assessment, and behavior control.
Thumbnail Image

China's 'surveillance creep': how big data COVID monitoring could be used to control people post-pandemic

2021-08-31
Qrius
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (big data and biometric-based health code monitoring) whose development and use have directly led to harms including privacy breaches, potential human rights violations, and coercive social control. The system's malfunction or misuse (e.g., data hacking, function creep to non-health purposes) has caused or is causing harm to individuals and communities. The article details realized harms rather than just potential risks, meeting the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

China's 'surveillance creep': how big data COVID monitoring could be used to control people post-pandemic " Stuff

2021-08-31
Stuff
Why's our monitor labelling this an incident or hazard?
The event involves AI systems in the form of big data analytics and automated health code risk assessments that directly influence citizen freedoms and privacy. The use and expansion of these AI systems have already led to realized harms, including privacy violations (data breaches) and social control measures that restrict rights and freedoms, constituting violations of human rights and harm to communities. Therefore, this qualifies as an AI Incident because the AI system's use has directly and indirectly led to significant harms as defined in the framework.
Thumbnail Image

MIL-Evening Report: China’s ‘surveillance creep’: how big data COVID monitoring could be used to...

2021-08-31
foreignaffairs.co.nz
Why's our monitor labelling this an incident or hazard?
The described big data COVID monitoring system involves AI systems analyzing personal and biometric data to assign health risk codes that control citizen movement and behavior. The system's expansion into mass surveillance and social credit scoring directly impacts fundamental rights and freedoms, constituting violations of human rights and harm to communities. The article documents realized harms such as privacy breaches, hacking incidents exposing biometric data, and coercive social control measures. Therefore, this event meets the criteria for an AI Incident due to direct harm caused by the AI system's use and function creep beyond its original purpose.