Should classification as an ACS-NSQIP high outlier be used to direct hospital quality improvement efforts?

ACS-NSQIP classifies hospitals as "high outliers" if their performance is significantly worse than expected. We determined how often hospitals return to as-expected performance after being newly identified as outliers.

Outlier status was identified in ACS-NSQIP semi-annual reports (SARs) 2008-2011 for 13 postoperative adverse events. Pearson correlation and R² measured the relationship between frequency of changes in outlier status, frequency of outlier identification, and adverse event rate.

Among 284 hospitals, 75% were classified as high outliers for an adverse event at least once. New high outliers frequently did not remain outliers in the next SAR. Of new outliers, mortality had the highest percentage return to as-expected performance (62.7%), while surgical site infection had the lowest (20.5%). The likelihood of an outlier hospital returning to as-expected performance was inversely related to the percentage of hospitals classified as outliers. The percentage of hospitals classified as outliers for an event explained 60% of variation in outlier hospitals returning to as-expected performance.

Outlier status may be less meaningful for adverse events with relatively few outlier hospitals.