Machine Learning on Medicare Claims Poorly Predicts the Individual Risk of 30-Day Unplanned Readmission After Total...

Unplanned hospital readmissions after total joint arthroplasty (TJA) represent potentially serious adverse events and remain a critical measure of hospital quality. Predicting the risk of readmission after TJA may provide patients and clinicians with valuable information for preoperative decision-making.

(1) Can nonlinear machine-learning models integrating preoperatively available patient, surgeon, hospital, and county-level information predict 30-day unplanned hospital readmissions in a large cohort of nationwide Medicare beneficiaries undergoing TJA? (2) Which predictors are the most important in predicting 30-day unplanned hospital readmissions? (3) What specific information regarding population-level associations can we obtain from interpreting partial dependency plots (plots describing, given our modeling choice, the potentially nonlinear shape of associations between predictors and readmissions) of the most important predictors of 30-day readmission?

National Medicare claims data (chosen because this database represents a large proportion of patients undergoing TJA annually) were analyzed for patients undergoing inpatient TJA between October 2016 and September 2018. A total of 679,041 TJAs (239,391 THAs [61.3% women, 91.9% White, 52.6% between 70 and 79 years old] and 439,650 TKAs [63.3% women, 90% White, 55.2% between 70 and 79 years old]) were included. Model features included demographics, county-level social determinants of health, prior-year (365-day) hospital and surgeon TJA procedure volumes, and clinical classification software-refined diagnosis and procedure categories summarizing each patient's Medicare claims 365 days before TJA. Machine-learning models, namely generalized additive models with pairwise interactions (prediction models consisting of both univariate predictions and pairwise interaction terms that allow for nonlinear effects), were trained and evaluated for predictive performance using area under the receiver operating characteristic (AUROC; 1.0 = perfect discrimination, 0.5 = no better than random chance) and precision-recall curves (AUPRC; equivalent to the average positive predictive value, which does not give credit for guessing "no readmission" when this is true most of the time, interpretable relative to the base rate of readmissions) on two holdout samples. All admissions (except the last 2 months' worth) were collected and split randomly 80%/20%. The training cohort was formed with the random 80% sample, which was downsampled (so it included all readmissions and a random, equal number of nonreadmissions). The random 20% sample served as the first test cohort ("random holdout"). The last 2 months of admissions (originally held aside) served as the second test cohort ("2-month holdout"). Finally, feature importances (the degree to which each variable contributed to the predictions) and partial dependency plots were investigated to answer the second and third research questions.

For the random holdout sample, model performance values in terms of AUROC and AUPRC were 0.65 and 0.087, respectively, for THA and 0.66 and 0.077, respectively, for TKA. For the 2-month holdout sample, these numbers were 0.66 and 0.087 and 0.65 and 0.075. Thus, our nonlinear models incorporating a wide variety of preoperative features from Medicare claims data could not well-predict the individual likelihood of readmissions (that is, the models performed poorly and are not appropriate for clinical use). The most predictive features (in terms of mean absolute scores) and their partial dependency graphs still confer information about population-level associations with increased risk of readmission, namely with older patient age, low prior 365-day surgeon and hospital TJA procedure volumes, being a man, patient history of cardiac diagnoses and lack of oncologic diagnoses, and higher county-level rates of hospitalizations for ambulatory-care sensitive conditions. Further inspection of partial dependency plots revealed nonlinear population-level associations specifically for surgeon and hospital procedure volumes. The readmission risk for THA and TKA decreased as surgeons performed more procedures in the prior 365 days, up to approximately 75 TJAs (odds ratio [OR] = 1.2 for TKA and 1.3 for THA), but no further risk reduction was observed for higher annual surgeon procedure volumes. For THA, the readmission risk decreased as hospitals performed more procedures, up to approximately 600 TJAs (OR = 1.2), but no further risk reduction was observed for higher annual hospital procedure volumes.

A large dataset of Medicare claims and machine learning were inadequate to provide a clinically useful individual prediction model for 30-day unplanned readmissions after TKA or THA, suggesting that other factors that are not routinely collected in claims databases are needed for predicting readmissions. Nonlinear population-level associations between low surgeon and hospital procedure volumes and increased readmission risk were identified, including specific volume thresholds above which the readmission risk no longer decreases, which may still be indirectly clinically useful in guiding policy as well as patient decision-making when selecting a hospital or surgeon for treatment.

Level III, therapeutic study.