Bias associated with mining electronic health records

Area of Science:

• Health Informatics
• Clinical Epidemiology
• Biostatistics

Background:

• Large-scale electronic health record (EHR) research presents unique challenges compared to traditional retrospective studies.
• Data quality issues, including inaccuracy, incompleteness, and complexity, can introduce significant biases into EHR-based research.
• Existing research highlights the potential for distorted results when relying solely on raw EHR data.

Purpose of the Study:

• To illustrate and quantify biases introduced by large-scale electronic health record research.
• To compare EHR research biases against a manually curated gold standard in a community-acquired pneumonia (CAP) study.
• To evaluate the impact of data quality and cohort selection on research outcomes.

Main Methods:

• Utilized a community-acquired pneumonia (CAP) study dataset with an established gold standard for comparison.
• Employed a naive approach using EHR data and subsequently refined the cohort through manual review.
• Assessed the influence of data errors and cohort characteristics on study results, particularly mortality estimates.

Main Results:

• A naive EHR data approach approximated the gold standard but was sensitive to errors in a minority of cases, substantially shifting mortality.
• Manual review identified errors in both cohort selection and characterization within the EHR data.
• Narrowing the study cohort based on manual review improved the accuracy of the results.

Conclusions:

• Large-scale EHR research requires rigorous methods to mitigate inherent data biases.
• Manual review and careful cohort refinement are essential for improving the reliability of EHR-based clinical research.
• While narrowing cohorts can enhance accuracy, it may introduce its own unquantifiable biases that warrant consideration.