When Stars Do Not Align: Overall Hospital Quality Star Ratings and the Volume-Outcome Association

Volume-outcome relationships are well established for coronary artery bypass grafting and total joint arthroplasty surgery. Although the U.S. Centers for Medicare & Medicaid Services (CMS) Overall Hospital Quality Star Ratings program includes outcome quality measures for these procedures, these outcome quality measures are not counted toward the star ratings for low-volume hospitals. We sought to assess whether excluding low-volume hospitals from surgical quality measures with known volume-outcome relationships affects the star ratings.

We identified quality measures used in CMS's star ratings that are related to surgical procedures with a known volume-outcome relationship and tested for the presence of the volume-outcome association for each of these measures. We then imputed missing values for low-volume hospitals for each measure and otherwise identically repeated the CMS calculations in order to assess the percentages of hospitals with the same, better, or worse ratings.

Among the measures used to calculate star ratings, we identified 4 quality measures (2 related to coronary artery bypass grafting and 2 related to total joint arthroplasty) with known volume-outcome relationships that were excluded from the calculations of the star ratings for low-volume hospitals. We confirmed a volume-outcome association in the CMS data for all 4 measures. When total joint arthroplasty complications were imputed for low-volume hospitals and then included in the calculation of the star ratings, over one-third of hospitals received a different rating; both low-volume and other hospitals were more often hurt than helped. Imputing the other 3 quality measures among low-volume hospitals left the ratings unchanged.

The CMS star ratings do not fully represent the risks of undergoing procedures at low-volume hospitals, potentially misrepresent quality across facilities, and hence are of uncertain utility to consumers.