Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Conformal Prediction Sets

by   Richard A. Berk, et al.

Risk assessment algorithms have been correctly criticized for potential unfairness, and there is an active cottage industry trying to make repairs. In this paper, we adopt a framework from conformal prediction sets to remove unfairness from risk algorithms themselves and the covariates used for forecasting. From a sample of 300,000 offenders at their arraignments, we construct a confusion table and its derived measures of fairness that are effectively free any meaningful differences between Black and White offenders. We also produce fair forecasts for individual offenders coupled with valid probability guarantees that the forecasted outcome is the true outcome. We see our work as a demonstration of concept for application in a wide variety of criminal justice decisions. The procedures provided can be routinely implemented in jurisdictions with the usual criminal justice datasets used by administrators. The requisite procedures can be found in the scripting software R. However, whether stakeholders will accept our approach as a means to achieve risk assessment fairness is unknown. There also are legal issues that would need to be resolved although we offer a Pareto improvement.


page 1

page 2

page 3

page 4


Almost Politically Acceptable Criminal Justice Risk Assessment

In criminal justice risk forecasting, one can prove that it is impossibl...

Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Optimal Transport and Conformal Prediction Sets

In the United States and elsewhere, risk assessment algorithms are being...

Nested Conformal Prediction Sets for Classification with Applications to Probation Data

Risk assessments to help inform criminal justice decisions have been use...

On (assessing) the fairness of risk score models

Recent work on algorithmic fairness has largely focused on the fairness ...

Fairness Deconstructed: A Sociotechnical View of 'Fair' Algorithms in Criminal Justice

Early studies of risk assessment algorithms used in criminal justice rev...

Feedback Effects in Repeat-Use Criminal Risk Assessments

In the criminal legal context, risk assessment algorithms are touted as ...

Risk Scores, Label Bias, and Everything but the Kitchen Sink

In designing risk assessment algorithms, many scholars promote a "kitche...

Please sign up or login with your details

Forgot password? Click here to reset