Statistical learning methods and cross-cultural fairness: Trade-offs and implications for risk assessment instruments

Area of Science:

• Forensic Psychology
• Statistical Learning
• Risk Assessment

Background:

• Statistical learning methods are increasingly used in risk assessment to enhance accuracy and discrimination (Area Under the Curve - AUC).
• Approaches to improve cross-cultural fairness in statistical learning are emerging but underexplored in forensic psychology, particularly in Australia.
• The Level of Service/Risk Needs Responsivity (LS/RNR) is a widely used risk assessment tool.

Purpose of the Study:

• To investigate the utility of statistical learning methods for improving both discrimination and cross-cultural fairness in risk assessment among Australian males.
• To evaluate the impact of preprocessing techniques on fairness metrics for Aboriginal and Torres Strait Islander compared to non-Aboriginal and Torres Strait Islander males.

Main Methods:

• Compared logistic regression, penalized logistic regression, random forest, stochastic gradient boosting, and support vector machine algorithms against the LS/RNR total risk score.
• Assessed discrimination using Area Under the Curve (AUC) and fairness through cross Area Under the Curve (xAUC), error rate balance, calibration, predictive parity, and statistical parity.
• Applied pre- and postprocessing techniques to statistical learning algorithms to enhance fairness.

Main Results:

• Statistical learning methods yielded comparable or slightly improved AUC values for discrimination.
• Processing approaches significantly improved several fairness metrics (xAUC, error rate balance, statistical parity) between Indigenous and non-Indigenous Australian males.
• The LS/RNR risk factors were utilized within the various machine learning algorithms.

Conclusions:

• Statistical learning methods, combined with processing approaches, offer a viable strategy for enhancing both discrimination and cross-cultural fairness in risk assessment instruments.
• Significant trade-offs exist between fairness and the application of statistical learning methods, necessitating careful consideration.
• Further research is warranted to explore the practical implementation and ethical implications of these methods in forensic psychology.