Longitudinal Reliability of Rheumatology Patient Safety Measures and Implications for CMS Case Minimums

Metrics with acceptable reliability are necessary to ensure that quality measures reward performance. Low metric reliability may result from a lack of differences between reporting entities or low numbers of eligible patients. The Merit-based Incentive Payment System (MIPS) incentivizes high-performing practices to report, which may lower practice-to-practice variation and lower reliability. This study evaluated the impact of voluntary participation in MIPS and measure denominator counts on practice-specific reliability estimates.

Data came from the Rheumatology Informatics System for Effectiveness (RISE) registry, which passively collects electronic health record (EHR) data. Practice-specific metric reliability was calculated on two safety measures-QPP176 (tuberculosis screening) and ACR10 (hepatitis B screening)-in two cohorts: the full sample of practices with data in RISE and the subset of practices that reported performance data on these measures to MIPS. The authors report longitudinal summary statistics on metric reliability, performance, and the number of eligible patients. The study team also examined estimates longitudinally stratified by patients counts.

Both the RISE and MIPS samples had acceptable median reliability (> 0.85) but a wide range (0.04-1.0) despite differences in performance. Analyses showed that for both measures for all years, the practice-specific metric reliability remains ≥ 0.80 when the number of eligible patients was at least 20 and ≥ 0.70 when the number of eligible patients was at least 10.

Self-selection into the MIPS program did not reduce the reliability of the examined patient safety measures. In addition, we found acceptable metric reliability at denominator counts lower than currently required by CMS suggesting case minimums for certain measures could be lowered to further encourage quality improvement in a wider subset of practices.