Improved American College of Surgeons NSQIP Hospital Benchmarking with Risk Adjustment for Many CPT Codes Rather Than...

Because of technical limitations inherent to logistic regression, NSQIP benchmarking has historically risk adjusted for procedure using only 1 principal CPT code among other predictors. This has the potential to create bias (favorable or unfavorable) for hospitals depending on how many multiple-procedure operations they do. Hospital quality assessments using current statistical methods were compared with those using a new methodology that permits risk adjustment incorporating many recorded CPT codes (capped here at 21).

American College of Surgeons NSQIP data from 2023, composed of 994,332 patients from 676 hospitals, were analyzed. Modeling included a preliminary logistic regression step where 5 years of historical data were used to generate a principal CPT code-specific linear risk score (logit) for each of 14 outcomes. This score is then used as one of many risk adjustment variables in follow-on models. For this reanalysis, the first step was replicated with a Catboost machine learning algorithm that provides a logit risk score based on a set of up to 21 CPT codes reported. Changes in hospital assessments using the 2 approaches to CPT code-based risk were examined.

Benchmarking results for the 14 outcomes were similar, but not identical, across the analytic methods. For 13 of 14 outcomes studied, the greater the mean number of CPT codes reported for patients in a hospital, the greater their benchmarking advantage when the model considered all codes; hospitals that reported only the principal CPT code had a benchmarking advantage when the model considered only that code.

Because of differences between hospitals in the proportion of multiple-procedure operations performed, risk adjustment using many CPT codes provides more defensible benchmarking results.