Spectral clustering identifies patterns of chiropractic care in a national longitudinal cohort

Characterise longitudinal patterns of chiropractic visits for neck pain or low back pain by using machine learning (ML) methods and explainable models.

Using de-identified claims data from 2016 to 2023 for adults from the Optum Labs Data Warehouse, we applied spectral clustering (SC) to identify novel patient clusters. Then we used explainable boosting machines (EBM) for feature ranking followed by hierarchical group lasso regression for feature selection. A logistic regression model used for parameter estimates.

SC identified 3 clusters-low, moderate and high dose-based on their pattern of chiropractic visits. An interesting finding was a small cluster where patients received persistently higher care for several months. Age, gender and number of prior visits to a chiropractor, primary care provider, or physical therapist emerged as strong indicators for provider type and frequency of visits.

Patients receiving spinal manipulative therapy sorted into 3 markedly different trajectories of utilisation. This unexpected variation mandates further investigation to identify optimal dose based on patient and provider characteristics. We also present EBM, a robust alternative to computationally heavy feature selection methods, to identify features necessary for predictive models. This approach obviates the need for opaque feature selection methods.

Results show the use of advanced, explainable methods to discover knowledge that can be missed by other methods. We present an approach to identify hidden patterns in large data that can guide hypothesis driven research. Our work can identify factors that drive high utilisation of services and inform practice guidelines.