A comparative approach of machine learning models to predict attrition in a diabetes management program

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Diabetes Self-Management

Background:

• Diabetes affects 11.6% of Americans, with high rates in South Carolina.
• Diabetes self-management programs improve weight, knowledge, and self-care.
• Retaining participants in these programs is crucial for managing the growing diabetic population.

Purpose of the Study:

• To evaluate machine learning (ML) methods for predicting attrition in a diabetes self-management program.
• To identify demographic, health, and spatial factors associated with program dropout.
• To assess the predictive performance of ML models using participant data.

Main Methods:

• Collected data from participants in the Health Extension for Diabetes (HED) program.
• Utilized descriptive statistics, Mann-Whitney U tests, and chi-square tests for demographic analysis.
• Applied various ML models (e.g., XGBoost) and SHAP for interpretability to predict attrition.

Main Results:

• Health-related measures (SF-12, DCI), demographics (race, age, height, education), and spatial variables (drive time) were influential predictors.
• ML models demonstrated poor overall performance (AUC 0.53-0.64, F1 0.19-0.36).
• XGBoost with downsampling showed the highest AUC (0.64) and F1 score (0.36) among tested models.

Conclusions:

• Current ML models are not sufficiently accurate for predicting individual attrition risk in diabetes self-management programs.
• Identified key factors influencing dropout, providing insights into participant engagement challenges.
• Further research is needed to enhance predictive modeling for improved patient retention in health programs.