The Progression of Disparities within the Criminal Justice System: Differential Enforcement and Risk Assessment Instruments

by   Miri Zilka, et al.

Algorithmic risk assessment instruments (RAIs) increasingly inform decision-making in criminal justice. RAIs largely rely on arrest records as a proxy for underlying crime. Problematically, the extent to which arrests reflect overall offending can vary with the person's characteristics. We examine how the disconnect between crime and arrest rates impacts RAIs and their evaluation. Our main contribution is a method for quantifying this bias via estimation of the amount of unobserved offenses associated with particular demographics. These unobserved offenses are then used to augment real-world arrest records to create part real, part synthetic crime records. Using this data, we estimate that four currently deployed RAIs assign 0.5–2.8 percentage points higher risk scores to Black individuals than to White individuals with a similar arrest record, but the gap grows to 4.5–11.0 percentage points when we match on the semi-synthetic crime record. We conclude by discussing the potential risks around the use of RAIs, highlighting how they may exacerbate existing inequalities if the underlying disparities of the criminal justice system are not taken into account. In light of our findings, we provide recommendations to improve the development and evaluation of such tools.


page 1

page 5

page 8

page 14

page 15

page 16

page 17


On the Validity of Arrest as a Proxy for Offense: Race and the Likelihood of Arrest for Violent Crimes

The risk of re-offense is considered in decision-making at many stages o...

Fairness Evaluation in Presence of Biased Noisy Labels

Risk assessment tools are widely used around the country to inform decis...

Risk-Efficient Bayesian Data Synthesis for Privacy Protection

High-utility and low-risks synthetic data facilitates microdata dissemin...

Experimental Evaluation of Algorithm-Assisted Human Decision-Making: Application to Pretrial Public Safety Assessment

Despite an increasing reliance on fully-automated algorithmic decision m...

Counterfactual Risk Assessments, Evaluation, and Fairness

Algorithmic risk assessments are increasingly used to help humans make d...

Ghosting the Machine: Judicial Resistance to a Recidivism Risk Assessment Instrument

Recidivism risk assessment instruments are presented as an 'evidence-bas...

Fair-by-design explainable models for prediction of recidivism

Recidivism prediction provides decision makers with an assessment of the...

Please sign up or login with your details

Forgot password? Click here to reset