Predicting the course of high-impact chronic pain using machine learning algorithms

High-impact chronic pain (HICP) affects over 17 million U.S. adults and follows highly variable courses. To date, the relative importance of biopsychosocial predictors of HICP incidence, persistence, and recovery, remains poorly understood. The National Health Interview Survey Longitudinal Cohort, which comprises 10,415 adults who completed baseline surveys in 2019 and follow-back surveys in 2020, is ideally suited for addressing this critical knowledge gap. Machine learning algorithms were evaluated for their discriminatory power and classification accuracy in predicting HICP group membership for eligible sample adults who reported pain at one or both time points (n=10,260): incident HICP (new onset in 2020; n=506), persistent HICP (present in both years; n=480), HICP recovery (present in 2019 only; n=471), or no HICP (neither year; n=8803). Shapley Additive Explanation values were generated for input features to assess their relative importance for model prediction. Gradient Boosting Decision Trees, which exhibited the highest discriminatory power (macro-AUC=0.80), revealed that family income was a strong predictor of both incident HICP and HICP recovery, while physical health (e.g., self-rated health, arthritis, prescription opioid use) was a key predictor of persistent HICP. Depression severity was the most important mental health predictor across all outcomes. This study highlights the relative prognostic importance of physical health, mental health, and socioeconomic factors for HICP over a one-year follow-up. These findings can help to advance predictive frameworks for HICP, refine clinical risk screening tools, and set the stage for future research evaluating modifiable risk and resilience factors to prevent or alleviate HICP. PERSPECTIVE: This study used a machine learning approach to reveal differential prognostic importance of biopsychosocial factors for high-impact chronic pain incidence, persistence, and recovery over a one-year follow-up. These findings could help improve clinical risk screening tools and inform future targeted interventions to prevent or alleviate high-impact chronic pain.