Related Experiment Video
Updated: May 24, 2025
Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model
Published on: August 16, 2020
Comparison of Machine Learning Algorithms Identifying Children at Increased Risk of Out-of-Home Placement:
Tyler J Gorham1, Rose Y Hardy2, David Ciccone2
1IT Research & Innovation, The Abigail Wexner Research Institute at Nationwide Children's Hospital, Columbus, Ohio, USA.
Insights
Machine learning models can identify children at risk of out-of-home placement. The eXtreme gradient-boosted trees (XGBoost) model showed better performance and less racial bias than the least absolute shrinkage and selection operator (LASSO) model.
Area of Science:
- Health Informatics
- Machine Learning in Healthcare
- Child Welfare Research
Background:
- Identifying children at risk of out-of-home placement is crucial for timely intervention.
- Medicaid-insured children represent a significant population facing potential out-of-home placement.
- Existing predictive models may not adequately address racial disparities.
Purpose of the Study:
- To develop and compare machine learning (ML) algorithms for identifying children at risk of out-of-home placement.
- To evaluate the performance of ML models with and without race as a predictor.
- To assess algorithmic bias in ML models for child welfare prediction.
Main Methods:
- Retrospective cohort study of Medicaid-insured children (2018-2022) in two Ohio counties.
- Development and comparison of least absolute shrinkage and selection operator (LASSO) and eXtreme gradient-boosted trees (XGBoost) ML algorithms.
- Performance evaluation using area under the receiver operating characteristic curve (AUROC) and partial AUROC (pAUROC90), with bias assessment across racial groups.
Main Results:
- XGBoost models demonstrated superior performance compared to LASSO models.
- XGBoost achieved an AUROC of 0.78 with race included and 0.76 without race.
- LASSO models achieved lower AUROCs (0.75 with race, 0.73 without race).
Conclusions:
- The eXtreme gradient-boosted trees (XGBoost) algorithm is more effective than LASSO for predicting out-of-home placement.
- The XGBoost model exhibited reduced evidence of racial bias compared to LASSO.
- Collaboration between ML developers and policy leaders is essential for creating equitable predictive models in child welfare.
Objective:
To develop a machine learning (ML) algorithm capable of identifying children at risk of out-of-home placement among a Medicaid-insured population.
Study Setting And Design:
The study population includes children enrolled in a Medicaid accountable care organization between 2018 and 2022 in two nonurban Ohio counties served by the Centers for Medicare and Medicaid Services-funded Integrated Care for Kids Model. Using a retrospective cohort, we developed and compared a set of ML algorithms to identify children at risk of out-of-home placement within one year. ML algorithms tested include least absolute shrinkage and selection operator (LASSO)-regularized logistic regression and eXtreme gradient-boosted trees (XGBoost). We compared both modeling approaches with and without race as a candidate predictor. Performance metrics included the area under the receiver operating characteristic curve (AUROC) and the corrected partial AUROC at specificities ≥ 90% (pAUROC90). Algorithmic bias was tested by comparing pAUROC90 across each model between Black and White children.
Data Sources And Analytic Sample:
The modeling dataset was comprised of Medicaid claims and patient demographics data from Partners For Kids, a pediatric accountable care organization.
Principal Findings:
Overall, XGBoost models outperformed LASSO models. When race was included in the model, XGBoost had an AUROC of 0.78 (95% confidence interval [CI]: 0.77-0.79) while the LASSO model had an AUROC of 0.75 (95% CI: 0.74-0.77). When race was excluded from the model, XGBoost had an AUROC of 0.76 (95% CI: 0.74-0.77) while LASSO had an AUROC of 0.73 (95% CI: 0.72-0.74).
Conclusions:
The more complex XGBoost outperformed the simpler LASSO in predicting out-of-home placement and had less evidence of racial bias. This study highlights the complexities of developing predictive models in systems with known racial disparities and illustrates what can be accomplished when ML developers and policy leaders collaborate to maximize data to meet the needs of children and families.
