FairLens detected racial bias in a recidivism prediction algorithm

FairLens, an open-source Python library that can uncover and measure data bias, was able to assess racial bias in the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) algorithm. This algorithm is used by judges and parole officers to assess the risk of recidivism for people with criminal convictions. The assessment using FairLens was partially inspired by a ProPublica investigation that found the COMPAS algorithm over-predicted recidivism for Black people and under-predicted it for White people. Since this investigation, other research has continued to find issues with the algorithm.

Using the COMPAS dataset, which includes the results from the COMPAS algorithm, FairLens identified the variables associated with sensitive attributes. Those variables corresponded with gender, age, and race. Using FairLens to visualize those attributes by risk score presented a clear picture that the largest disparity across scores was in fact race. In other words, as the risk score increased the amount of Black people also increased, while the amount of White people decreased. FairLens metrics then provided a more precise measurement of the statistical distance of the risk score’s distribution in each subgroup and in the entire dataset. The FairLens fairness scorer can then use the relevant hypothesis tests to determine the differences in risk score distributions by each subgroup, like race. Finally, FairLens was able to find that even when dropping race as an attribute, predictions of recidivism remain biased toward people of different racial groups. 

Tutorial: FairLens tutorial for COMPAS Recidivism dataset 

Dataset: COMPAS dataset

FairLens' sensitive attribute and proxy detection plays an especially relevant role in this use case. It provides a relatively straightforward workflow to uncover systemic bias reflected in a dataset, which allows developers without deep subject matter expertise in the criminal justice system to still provide credible insights related to bias.

FairLens plays a useful role in detecting and measuring bias, but additional context for how bias persists in the criminal justice system is required to provide rich explanations for why bias occurred based on race, especially if those potential explanations are not referenceable in the dataset.

Use the following components in the FairLens library if you want to conduct a similar analysis:

fairlens.sensitive: Sensitive attribute and proxy detection 

fairlens.metrics: Collection of distance, correlation metrics and statistical tests

fairlens.plot: Tools to visualize distributions and heatmaps 

fairlens.scorer: Automatically generate a fairness report for a dataset

This article does not constitute legal or other professional advice and was written by students as part of the Duke Ethical Technology Practicum.