Comparison of Machine Learning Algorithms Identifying Children at Increased Risk of Out-of-Home Placement:...

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Child Welfare Research

Background:

• Identifying children at risk of out-of-home placement is crucial for timely intervention.
• Medicaid-insured children represent a significant population facing potential out-of-home placement.
• Existing predictive models may not adequately address racial disparities.

Purpose of the Study:

• To develop and compare machine learning (ML) algorithms for identifying children at risk of out-of-home placement.
• To evaluate the performance of ML models with and without race as a predictor.
• To assess algorithmic bias in ML models for child welfare prediction.

Main Methods:

• Retrospective cohort study of Medicaid-insured children (2018-2022) in two Ohio counties.
• Development and comparison of least absolute shrinkage and selection operator (LASSO) and eXtreme gradient-boosted trees (XGBoost) ML algorithms.
• Performance evaluation using area under the receiver operating characteristic curve (AUROC) and partial AUROC (pAUROC90), with bias assessment across racial groups.

Main Results:

• XGBoost models demonstrated superior performance compared to LASSO models.
• XGBoost achieved an AUROC of 0.78 with race included and 0.76 without race.
• LASSO models achieved lower AUROCs (0.75 with race, 0.73 without race).

Conclusions:

• The eXtreme gradient-boosted trees (XGBoost) algorithm is more effective than LASSO for predicting out-of-home placement.
• The XGBoost model exhibited reduced evidence of racial bias compared to LASSO.
• Collaboration between ML developers and policy leaders is essential for creating equitable predictive models in child welfare.