The Fault in Our Sets: A Mixed Methods Analysis of Clinical Value Set Errors

To characterize clinical value set issues and identify common patterns of errors.

We conducted semi-structured interviews with 26 value set experts and performed root cause analyses of errors identified in electronic health records (EHRs). We also analyzed a random sample of user-reported issues from the Value Set Authority Center (VSAC), developing a categorization scheme for value set errors. Additionally, we audited medication value sets from three sources and assessed the impact of value set variations on a clinical quality measure within Vanderbilt's Epic system.

Interviews highlighted ongoing difficulties in value set identification, creation, and maintenance, with significant consequences for clinical decision support (CDS), quality measurement, and patient care. Content analysis indicated that 42% of errors involved missing codes, 14% included extraneous codes, and 40% arose from misinterpretations of value set intent; 72% of these errors were present at creation. The audit revealed errors in 50% of medication value sets, predominantly omissions. The impact analysis demonstrated that value set selection altered a clinical quality measure's outcome by 3- to 30-fold.

Value set errors are widespread and arise from a delineable set of causes. Characterizing patterns of errors allowed us to identify best practices and potential solutions to minimize their frequency.

Better tools for finding, authoring, auditing and monitoring value sets are urgently needed.