Can Machine Learning Improve Screening for Targeted Delinquency Prevention Programs?

Area of Science:

• Forensic Psychology
• Developmental Psychology
• Biostatistics

Background:

• Effective delinquency prevention programs rely on accurate screening of at-risk youth.
• Current screening methods often lack accuracy, leading to resource inefficiency and missed intervention opportunities.

Purpose of the Study:

• To evaluate the accuracy of logistic regression and machine learning algorithms for screening 5th-grade boys at risk of future serious and violent offenses.
• To compare the predictive performance of these advanced methods against traditional sum-score approaches.

Main Methods:

• Developed screening algorithms using teacher-reported externalizing problems and other risk factors.
• Compared predictive performance using Area Under the Receiver Operating Curve (AUROC) and Brier score on holdout data.
• Evaluated logistic regression, machine learning, and traditional sum-score methods.

Main Results:

• Both logistic regression and machine learning models demonstrated superior AUROC compared to sum-score methods when considering a broad range of risk factors.
• The accuracy improvement was modest and diminished when using item-level data for externalizing problems.
• Machine learning algorithms did not outperform simple logistic models after appropriate cross-validation.

Conclusions:

• Logistic regression screening may enhance the cost-effectiveness of delinquency prevention programs in specific contexts.
• Currently, machine learning offers no discernible marginal benefit over logistic regression for this screening purpose.
• Rigorous cross-validation is crucial for accurately assessing the performance of predictive algorithms.