What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask

Area of Science:

• Health Informatics
• Medical Data Science
• Real-World Evidence Research

Background:

• The COVID-19 pandemic spurred a rise in studies using electronic health record (EHR) data.
• Several high-profile EHR studies faced retractions due to data quality and methodological concerns.
• Many medical professionals and editors lack a framework for critically evaluating EHR-derived research.

Purpose of the Study:

• To provide a framework for critically appraising studies that utilize electronic health record (EHR) data.
• To highlight key considerations for researchers, clinicians, and stakeholders reviewing EHR-derived studies.
• To promote best practices for ensuring the rigor and reliability of EHR-based research.

Main Methods:

• Distillation of six crucial considerations from the broader informatics literature.
• Focus on key aspects for evaluating EHR data and study methodologies.
• Identification of best practices for manuscript and grant review.

Main Results:

• Six key considerations for appraising EHR data studies were identified: data completeness, data collection/handling, data type (codified vs. textual), methodological robustness against EHR variability, transparency of data/code, and multidisciplinary approach.
• These considerations address the inherent challenges and opportunities in EHR data analysis.
• The framework aims to guide critical appraisal beyond conventional statistical methods.

Conclusions:

• A structured approach is essential for evaluating the quality of EHR-derived studies.
• Adherence to these six considerations will enhance the rigor and reliability of real-world evidence from EHRs.
• Promoting critical appraisal fosters confidence and advances the field of health informatics.