Using tree-based ensemble methods to produce a population-based mortality risk score in Ontario, Canada.

Area of Science:

• Epidemiology
• Machine Learning
• Health Services Research

Background:

• Risk adjustment is crucial in observational epidemiology to mitigate confounding.
• Accurate outcome prediction, such as mortality, is key for effective risk adjustment.
• This study compares logistic regression with tree-based ensemble methods for mortality prediction.

Purpose of the Study:

• To compare the performance of various machine learning models against logistic regression for predicting 1-year mortality in the general population of Ontario, Canada.
• To identify the most accurate predictive model for enhancing risk adjustment in observational studies.
• To assess the impact of detailed comorbidity definitions on predictive model performance.

Main Methods:

• Utilized administrative health data from Ontario adults (age ≥18) alive as of January 1, 2022.
• Applied logistic regression, random forests, extremely randomized trees, adaptive boosting, gradient boosting, extreme gradient boosting, Newton boosting, and CatBoost to predict 1-year mortality.
• Evaluated models using Area Under the ROC Curve (AUROC), Precision-Recall AUC (PR-AUC), Brier score, and Integrated Calibration Index (ICI) on a 30% test set.

Main Results:

• Included 12,080,801 individuals; 1.0% died within 1 year.
• CatBoost achieved the highest discrimination (AUROC 0.933, PR-AUC 0.280) and calibration (ICI 0.0003), outperforming logistic regression (AUROC 0.926, PR-AUC 0.256).
• Detailed comorbidity definitions (cancer subtypes, chronic kidney disease via serum creatinine) improved PR-AUC and calibration for high-risk individuals. Age was the most influential feature.

Conclusions:

• The CatBoost machine learning model demonstrated superior accuracy in predicting 1-year mortality for the general Ontario population.
• Machine learning methods, exemplified by CatBoost, can significantly improve risk adjustment in observational studies.
• Enhanced risk adjustment facilitates more accurate confounder control, supporting health policy and epidemiologic research.