Maintaining automated measurement of Choosing Wisely adherence across the ICD 9 to 10 transition

It remains unclear how to incorporate terminology changes, such as the International Classification of Disease (ICD) transition from ICD-9 to ICD-10, into established automated healthcare quality metrics.

To evaluate whether general equivalence mapping (GEM) can apply ICD-9 based metrics to ICD-10 patient data. To develop and validate novel ICD-10 reference codesets.

Retrospective analysis for eleven Choosing Wisely (CW) metrics was performed using three scripted algorithms on an institutional clinical data warehouse. ICD-10 data were compared against published ICD-9 based metric definitions using two equivalence mapping algorithms. A third algorithm implemented novel reference ICD-10 codes matching the original ICD-9 codes' intent for comparison with patient ICD-10 data.

All adult patients seen at Vanderbilt University Medical Center, April - September 2016.

The prevalence of eleven CW services during the six-month period.

The three algorithms found similar prevalence of avoidable CW services, with an unweighted-mean of 8.4% (range: 0.16-65%), or approximately 20,000 CW services out of 240,000 potential cases in 515,406 unique patients. The algorithms' median sensitivity was 0.80 (interquartile range: 0.75-0.95), median specificity was 0.88 (IQR: 0.77-0.94), and median Rand accuracy was 0.84 (IQR: 0.79-0.89). The attributed waste of these eleven services for the period ranged from $871,049 to $951,829 between methods. Accuracy assessment demonstrated that the GEM-based methods suffered recall losses for metrics requiring multistep mapping due to incompleteness, while novel ICD-10 metric definitions avoided these challenges.

Comprehensive mapping enables use of legacy metrics across ICD generations, but requires computational complexity that can be avoided with novel ICD-10 based metric definitions. Variation in the dollars attributed to waste due to ICD mapping introduces ambiguity that may affect quality-based reimbursement.