The accuracy, fairness, and limits of predicting recidivism

Area of Science:

• Criminology
• Computer Science
• Machine Learning

Background:

• Risk assessment algorithms are increasingly used in criminal justice.
• These tools, like COMPAS, are claimed to be more accurate and less biased than human judgment due to big data and machine learning.
• Their application spans pretrial, parole, and sentencing decisions.

Purpose of the Study:

• To evaluate the accuracy and fairness of the COMPAS risk assessment tool.
• To compare the performance of COMPAS against human predictions and simpler models.
• To investigate the impact of feature complexity on recidivism prediction.

Main Methods:

• Comparative analysis of COMPAS predictions against actual recidivism rates.
• Evaluation of prediction accuracy and fairness metrics for COMPAS.
• Development and assessment of a simple linear predictor using limited features.
• Comparison of the simple model's performance against COMPAS.

Main Results:

• COMPAS demonstrated no superior accuracy or fairness compared to predictions from individuals with minimal criminal justice experience.
• A basic linear predictor utilizing only two features achieved performance comparable to the 137-feature COMPAS system.
• The study challenges the purported advantages of complex machine learning in recidivism prediction.

Conclusions:

• The effectiveness of sophisticated algorithms like COMPAS in predicting criminal recidivism is questionable.
• Simpler predictive models may offer comparable or superior performance with reduced complexity and potential bias.
• Further research is needed to ensure fairness and accuracy in algorithmic decision-making within the criminal justice system.