Rotterdam Welfare Fraud Algorithm Risks Discrimination and Bias

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The Rotterdam municipality's use of AI algorithms to detect welfare fraud has led to biased outcomes, disproportionately targeting individuals with migration backgrounds due to proxy variables. Reports highlight insufficient transparency, oversight, and responsibility, raising concerns about discrimination, profiling, and unfair treatment of citizens.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly discusses algorithms used by the municipality to assess welfare fraud risk, which qualifies as AI systems due to their predictive and decision-influencing nature. The report highlights risks of discrimination and profiling stemming from the algorithm's use of proxy variables that indirectly relate to protected characteristics, which could lead to violations of rights and unfair treatment. However, the report states that the actual operation and impact of the algorithms have not been fully investigated, and no confirmed incidents of harm are described. Therefore, the event fits the definition of an AI Hazard, as the use of these AI systems could plausibly lead to discrimination and profiling harms in the future if unaddressed.[AI generated]
AI principles
AccountabilityFairnessTransparency & explainabilityRespect of human rightsDemocracy & human autonomy

Industries
Government, security, and defence

Affected stakeholders
Other

Harm types
Human or fundamental rights

Business function:
Compliance and justice

AI system task:
Event/anomaly detection


Articles about this incident or hazard