The Fault in Our Sets: A Mixed Methods Analysis of Clinical Value Set Errors

Area of Science:

• Health Informatics
• Clinical Data Management
• Healthcare Quality Improvement

Background:

• Clinical value sets are essential for standardizing data in electronic health records (EHRs).
• Errors in value sets can negatively impact clinical decision support (CDS), quality measurement, and patient care.
• Existing challenges in value set identification, creation, and maintenance require systematic investigation.

Purpose of the Study:

• To characterize the nature and frequency of clinical value set issues.
• To identify common patterns and root causes of errors in clinical value sets.
• To assess the impact of value set variations on clinical quality measures.

Main Methods:

• Conducted semi-structured interviews with 26 value set experts.
• Performed root cause analyses of errors in EHRs.
• Analyzed user-reported issues from the Value Set Authority Center (VSAC).
• Audited medication value sets from multiple sources.
• Assessed the impact of value set variations on a clinical quality measure.

Main Results:

• 42% of errors involved missing codes, 14% extraneous codes, and 40% misinterpretation of intent.
• 72% of errors were present at the time of value set creation.
• 50% of audited medication value sets contained errors, primarily omissions.
• Value set selection variations altered clinical quality measure outcomes by 3- to 30-fold.

Conclusions:

• Value set errors are widespread and stem from identifiable causes.
• Characterizing error patterns can inform best practices and solutions.
• There is an urgent need for improved tools for value set authoring, auditing, and monitoring.