DWP's AI Fraud Detection System Risks Unfair Denial of Universal Credit Benefits

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The UK Department for Work and Pensions is trialling a machine learning algorithm to detect potential Universal Credit fraud. Campaigners warn the system could unfairly penalise vulnerable groups, with the DWP admitting some legitimate claims may be wrongly flagged, leading to wrongful denial of benefits and financial harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly states that a machine learning algorithm is used to predict fraud in benefit claims, influencing decisions on whether claimants receive universal credit. Campaigners and the National Audit Office highlight risks of biased outcomes and discrimination against protected groups, which are violations of human rights and labor rights. The harm is realized or ongoing, as benefits could be stopped before payment based on the algorithm's output. The AI system's development and use have directly led to these harms, meeting the criteria for an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
FairnessAccountabilityTransparency & explainabilityHuman wellbeingRespect of human rightsRobustness & digital security

Industries
Government, security, and defence

Affected stakeholders
Consumers

Harm types
Economic/PropertyPsychologicalReputationalHuman or fundamental rights

Business function:
Compliance and justiceMonitoring and quality control

AI system task:
Event/anomaly detection


Articles about this incident or hazard