Are ICD codes reliable for observational studies? Assessing coding consistency for data quality.

Area of Science:

• Health Informatics
• Medical Coding Systems
• Electronic Health Records (EHRs)

Background:

• International Classification of Diseases (ICD) codes are crucial for patient cohort creation and phenotype definition in EHRs.
• Inconsistent ICD code assignment can compromise the utility and reliability of research findings derived from EHR data.

Purpose of the Study:

• To assess the reliability of ICD code assignment during the transition from ICD-9-CM to ICD-10-CM.
• To investigate temporal and geographical variations in ICD code usage within a US health system.

Main Methods:

• Utilized General Equivalence Mapping (GEM) tables to cluster equivalent ICD codes.
• Employed deep learning and statistical models to analyze EHR data from the US Veterans Administration Central Data Warehouse.
• Examined changes in ICD code assignments across the transition period and at individual VA facilities.

Main Results:

• A significant number of frequently used ICD code clusters exhibited substantial deviations between ICD-9-CM and ICD-10-CM assignments.
• Manual review revealed problematic changes in 66% of sampled code clusters, with 37% lacking clear explanations.
• Observed coding patterns varied considerably across different care locations.

Conclusions:

• The variability in ICD code assignment across time and location raises concerns about the semantic reliability of EHR-based cohorts and phenotypes.
• Researchers must carefully consider and define cohort selection and phenotype definitions due to observed coding inconsistencies.
• The transition to ICD-10-CM highlighted underlying challenges in maintaining consistent ICD code usage in EHRs.