Optimized Risk Assessment in Forensic Practice: A Comparison of Machine Learning and Manual Scoring Approaches

Area of Science:

• Criminology
• Forensic Psychology
• Data Science

Background:

• Correctional jurisdictions and risk assessment developers seek optimal scoring methods.
• Existing comparisons focus on predictive validity, overlooking calibration and item weighting.
• Machine learning algorithms are increasingly explored for risk assessment tools.

Purpose of the Study:

• To compare manual and machine learning scoring methods for risk assessment tools.
• To evaluate predictive validity, calibration, item inclusion, and item weighting.
• To determine the optimal scoring method for the Reduction in Capacity Evaluation (ReduCE) tool.

Main Methods:

• Developed scoring methods for the ReduCE tool using manual (unweighted, Burgess, Nuffield, Nuffield 2.5, regression) and machine learning (artificial neural network, random forests) approaches.
• Compared methods based on predictive validity, calibration, item inclusion, and item weighting.
• Assessed drawbacks associated with each scoring approach.

Main Results:

• Machine learning methods did not outperform manual methods in predictive validity or calibration.
• Machine learning approaches introduced drawbacks concerning item inclusion and weighting.
• The Nuffield 2.5 manual scoring method demonstrated optimal performance for the ReduCE tool.

Conclusions:

• Manual scoring methods, particularly Nuffield 2.5, are preferable to machine learning for the ReduCE risk assessment tool.
• Comprehensive evaluation beyond predictive validity is crucial for selecting optimal risk assessment scoring methods.
• The findings inform correctional jurisdictions and developers in optimizing risk assessment instruments.