Predictive Policing AI Leads to Harm Through Racial Bias in Chicago

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

In 2013, Chicago resident Robert McDaniel was repeatedly targeted by police and social workers after a predictive policing AI flagged him as likely to be involved in a shooting, despite only minor offenses. The AI's biased outputs led to increased surveillance and harm, highlighting the dangers of algorithmic discrimination.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly describes an AI system used in predictive policing that led to increased police attention on a Black individual, which indirectly resulted in harm (being shot twice). This constitutes an AI Incident because the AI system's use directly contributed to violations of human rights and harm to a person. The discussion of systemic bias and real-world consequences aligns with the definition of AI Incident, as the AI's outputs have materially harmed individuals and communities. Therefore, the event is classified as an AI Incident.[AI generated]
AI principles
FairnessPrivacy & data governanceRespect of human rightsTransparency & explainabilityAccountabilitySafetyHuman wellbeingDemocracy & human autonomy

Industries
Government, security, and defence

Affected stakeholders
General public

Harm types
Human or fundamental rightsPsychologicalReputationalPublic interest

Severity
AI incident

Business function:
Compliance and justice

AI system task:
Forecasting/prediction


Articles about this incident or hazard

Thumbnail Image

Can a machine be racist? Artificial Intelligence has shown troubling signs of bias, but there are reasons for optimism

2023-03-06
The Conversation
Why's our monitor labelling this an incident or hazard?
The article explicitly describes an AI system used in predictive policing that led to increased police attention on a Black individual, which indirectly resulted in harm (being shot twice). This constitutes an AI Incident because the AI system's use directly contributed to violations of human rights and harm to a person. The discussion of systemic bias and real-world consequences aligns with the definition of AI Incident, as the AI's outputs have materially harmed individuals and communities. Therefore, the event is classified as an AI Incident.
Thumbnail Image

Can a machine be racist? AI has shown troubling signs of bias, but

2023-03-07
Deccan Herald
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (predictive policing algorithms) whose use indirectly led to harm (shootings linked to surveillance and targeting). However, the article is primarily an analytical and reflective piece discussing known issues of AI bias and systemic harm rather than reporting a new discrete incident or hazard event. It synthesizes existing knowledge and critiques, contributing to the broader understanding of AI harms and societal responses. Therefore, it fits best as Complementary Information, enhancing understanding of AI-related harms without describing a new incident or hazard.
Thumbnail Image

Can a machine be racist? Artificial Intelligence has shown troubling signs of bias, but there are reasons for optimism | Education

2023-03-07
Devdiscourse
Why's our monitor labelling this an incident or hazard?
The article explicitly references an AI system used for predictive policing that directly contributed to harm against a person (Robert McDaniel), including increased police attention and eventual shootings. This constitutes an AI Incident because the AI system's use directly led to harm to a person and exacerbated social inequalities. The article also discusses other examples of AI bias causing harm in various domains, reinforcing the classification as an AI Incident rather than a hazard or complementary information. The focus is on realized harms caused or exacerbated by AI systems, not just potential or future risks.
Thumbnail Image

Can a machine be racist? AI has shown troubling signs of bias, but there are reasons for optimism

2023-03-07
Tech Xplore
Why's our monitor labelling this an incident or hazard?
The article explicitly describes an AI system (predictive policing software) whose use directly led to harm to a person (Robert McDaniel) through increased police attention and subsequent shootings. This constitutes harm to a person (a) and violations of rights (c) due to biased AI outputs influenced by historical and social inequalities. The AI system's role is pivotal in causing these harms, fulfilling the criteria for an AI Incident. The article also discusses systemic bias in AI more generally, but the concrete example of harm makes this an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Can AI Be Racist? Signs Of Bias Seen But There Are Reasons For Optimism

2023-03-07
english
Why's our monitor labelling this an incident or hazard?
The article explicitly references an AI system used for predictive policing that directly influenced police and social worker actions towards an individual, which indirectly led to harm (shootings). This meets the definition of an AI Incident as the AI system's use has indirectly led to harm to a person. Additionally, the article discusses broader harms caused by AI bias in various domains, reinforcing the classification. The AI system's involvement is in its use, and the harm is realized, not just potential. Hence, the classification as AI Incident is appropriate.