Comparison of machine learning approaches for positive airway pressure adherence prediction in a veteran cohort

Adherence to positive airway pressure (PAP) therapy for sleep apnea is suboptimal, particularly in the veteran population. Accurately identifying those best suited for other therapy or additional interventions may improve adherence. We evaluated various machine learning algorithms to predict 90-day adherence.

The cohort of VA Northeast Ohio Health Care system patients who were issued a PAP machine (January 1, 2010-June 30, 2015) had demographics, comorbidities, and medications at the time of polysomnography obtained from the electronic health record. The data were split 60:20:20 into training, calibration, and validation data sets, with no use of validation data for model development. We constructed models for the first 90-day adherence period (% nights ≥4 h use) using the following algorithms: linear regression, least absolute shrinkage and selection operator, elastic net, ridge regression, gradient boosted machines, support vector machine regression, Bayes-based models, and neural nets. Prediction performance was evaluated in the validation data set using root mean square error (RMSE).

The 5,047 participants were 38.3 ± 11.9 years old, and 96.1% male, with 36.8% having coronary artery disease and 52.6% with depression. The median adherence was 36.7% (interquartile range: 0%, 86.7%). The gradient boosted machine was superior to other machine learning techniques (RMSE 37.2). However, the performance was similar and not clinically useful for all models without 30-day data. The 30-day PAP data and using raw diagnoses and medications (vs. grouping by type) improved the RMSE to 24.27.

Comparing multiple prediction algorithms using electronic medical record information, we found that none has clinically meaningful performance. Better adherence predictive measures may offer opportunities for personalized tailoring of interventions.