Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Optimal Transport and Conformal Prediction Sets

Bibliographic Data

ID2331148
AuthorsRichard A Berk (0000-0002-2983-1276, University of Pennsylvania, Philadelphia, PA, USA, corresponding author), Arun Kumar Kuchibhotla (0000-0003-4459-5352, Carnegie Mellon University, Pittsburgh, PA, USA, corresponding author), Eric Tchetgen Tchetgen (University of Pennsylvania, Philadelphia, PA, USA)
Year2024
Volume53
Issue4
Pages1629-1675
Publication date2024-11-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueSociological Methods & Research (JOURNAL)
Journal identifiersISSN: 0049-1241 • E-ISSN: 1552-8294
PublisherSAGE Publications Inc (PUBLISHER)
DOI10.1177/00491241231155883
OpenAlexW3212296934
LanguageEN
Citations received1
References cited47

In the United States and elsewhere, risk assessment algorithms are being used to help inform criminal justice decision-makers. A common intent is to forecast an offender's 'future dangerousness.' Such algorithms have been correctly criticized for potential unfairness, and there is an active cottage industry trying to make repairs. In this paper, we use counterfactual reasoning to consider the prospects for improved fairness when members of a disadvantaged class are treated by a risk algorithm as if they are members of an advantaged class. We combine a machine learning classifier trained in a novel manner with an optimal transport adjustment for the relevant joint probability distributions, which together provide a constructive response to claims of bias-in-bias-out. A key distinction is made between fairness claims that are empirically testable and fairness claims that are not. We then use confusion tables and conformal prediction sets to evaluate achieved fairness for estimated risk. Our data are a random sample of 300,000 offenders at their arraignments for a large metropolitan area in the United States during which decisions to release or detain are made. We show that substantial improvement in fairness can be achieved consistently with a Pareto improvement for legally protected classes

Actuarial science · Consistency (knowledge bases) · Constructive · Counterfactual thinking · Criminal justice · Criminology · Disadvantaged · Econometrics · Economics · Operations management · Pareto principle · Political science · Artificial Intelligence · Computer Science · Criminal Justice and Corrections Analysis · Ethics and Social Impacts of AI · Law · Law, Economics, and Judicial Systems · Psychology · Social Psychology

Tools

Open DOIOpen Access