Empirical Likelihood-Based Fairness Auditing

The Empirical Likelihood-Based Fairness Auditing framework is a statistical method for auditing the fairness of machine learning (ML) models. ML models used in high-stakes decisions, such as predicting reoffending or selecting job candidates, often perform differently for different demographic groups, raising concerns about algorithmic bias. Fairness auditing addresses this risk in two ways. Certification verifies whether a model meets fairness constraints. Flagging identifies specific groups that are treated differently. Existing auditing techniques often rely on restrictive assumptions about how data is distributed, or require heavy computation. The framework addresses both limitations.

The framework is based on empirical likelihood, a statistical technique that does not assume any particular data distribution. It builds statistics that measure differences in model performance between groups. These statistics follow well-known chi-square or mixed chi-square distributions as sample sizes grow. This allows reliable confidence intervals and hypothesis tests without distributional assumptions. The underlying optimisation is numerically stable, which supports both large-scale certification and efficient discovery of affected subgroups.

In experiments, the authors report that the method produces more accurate confidence intervals than bootstrap-based approaches. It is also faster by several orders of magnitude. Applied to the COMPAS dataset, it flags intersectional biases. For example, it finds a significantly higher rate of positive predictions for African-American men under 25, and systematic under-prediction for Caucasian women, compared with the population average.

Auto-discovered on 2026-02-11 by OECD Catalogue Automation