Dynamic Updating Strategies to Assess Hospital Performance of Surgical Aortic Valve Replacement

Prediction models determining expected outcomes are infrequently updated (ie, static), which may reduce accuracy and misclassify hospital performance over time. Dynamic models incorporate changes over time and may improve accuracy and fairness in hospital comparisons. This study evaluated whether dynamic updating, compared with a static model, altered hospital rankings and outlier detection among surgical aortic valve replacement patients.

This retrospective cohort study assessed performance across 53 hospitals using claims data from the Pennsylvania Health Care Cost Containment Council. A multivariable logistic regression model using clinical and demographic variables was developed on data from 1999 to 2006 to predict 30-day postoperative mortality, then applied to testing data from 2007 to 2018 to compare 4 strategies: (1) a static model with fixed parameters, (2) an annual correction factor based on The Society of Thoracic Surgeons methodology, (3) calibration regression for annual recalibration, and (4) dynamic logistic state space model to continuously update model coefficients. Performance was evaluated using observed-to-expected ratios and Z scores. Lower values indicate better-than-expected outcomes.

The training sample included 14 070 patients (mean age 66.6; 43.1% women); the testing sample included 29 127 patients (mean age 67.4; 39.1% women). The static model had the widest Z score variability (range -6.97 to 1.38), compared with calibration regression (-3.04 to 2.85), correction factor (-2.87 to 3.24), and dynamic logistic state space model (-2.57 to 3.03). The static model labeled 15 hospitals as significantly better-than-expected; only 3 (20.0%) maintained this classification with the correction factor and dynamic logistic state space model, and 5 (33.3%) with calibration regression. No hospitals were classified as significantly worse-than-expected under the static model, whereas calibration regression identified 6, and both dynamic logistic state space model and the correction factor identified 7.

Static models may misclassify hospital performance and rankings. Dynamic strategies influence outlier detection and change hospital rankings over time. Regular model updates may better reflect current performance, supporting fairer hospital comparisons.