Does machine learning improve prediction of VA primary care reliance?

Area of Science:

• Health Services Research
• Health Informatics
• Biostatistics

Background:

• The Veterans Affairs (VA) Health Care System serves a large population, many of whom also use Medicare.
• Predicting future reliance on VA services for dual-use patients is crucial for resource allocation.
• Limited research exists on advanced methods for predicting veteran healthcare utilization patterns.

Purpose of the Study:

• To evaluate the efficacy of machine learning models in predicting future reliance on VA primary care among dual-use veterans.
• To compare the predictive performance of machine learning techniques against traditional statistical methods.

Main Methods:

• An observational study analyzed data from 83,143 dual-enrolled VA and Medicare patients.
• Six models (logistic regression, elastic net, decision trees, random forest, gradient boosting, neural network) were compared.
• Performance was measured using the area under the receiver operating characteristic (AUROC) curve to predict >50% VA primary care visits.

Main Results:

• Overall, 72.9% and 74.5% of veterans were primarily VA-reliant in 2012 and 2013, respectively.
• All models demonstrated comparable average AUROCs, ranging from 0.873 to 0.892.
• Gradient boosting machine showed a modest improvement in AUROC over standard logistic regression.

Conclusions:

• Advanced machine learning models offer minimal performance improvement over logistic regression for predicting veteran VA primary care reliance.
• The computational complexity and reduced interpretability of models like gradient boosting may outweigh their marginal benefits.
• Traditional statistical methods remain a practical choice for predicting healthcare utilization in this population.