France Travail's AI Profiling System Raises Concerns Over Discrimination and Rights Violations

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

France Travail is testing an AI algorithm to profile and select unemployed individuals for increased controls and potential sanctions. The system, criticized by advocacy groups for bias and lack of transparency, has led to higher control rates and raised concerns about discrimination, stigmatization, and violations of labor and human rights in France.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly mentions an AI system developed and used by France Travail to profile and target job seekers for control, based on predictive analysis from past data. This AI-driven profiling can lead to violations of human rights and labor rights by automating surveillance and potentially unfairly targeting individuals. The AI system's use in this context directly leads to harm through invasive monitoring and possible unjust treatment of job seekers. Therefore, this event qualifies as an AI Incident due to the realized harm linked to the AI system's use in surveillance and control.[AI generated]
AI principles
FairnessTransparency & explainability

Industries
Government, security, and defence

Affected stakeholders
General public

Harm types
Human or fundamental rightsPsychological

Severity
AI incident

Business function:
Compliance and justice

AI system task:
Forecasting/prediction


Articles about this incident or hazard