Assessing document section heterogeneity across multiple electronic health record systems for computational...

Area of Science:

• Health Informatics
• Computational Linguistics
• Machine Learning

Background:

• Clinical narrative interpretation is vital for computational phenotyping.
• Context, particularly clinical section information, is key for accurate term interpretation.
• Electronic Health Record (EHR) system heterogeneity challenges the use of section information.

Purpose of the Study:

• To quantitatively assess heterogeneity in EHR section information.
• To evaluate machine learning (ML) classifiers for mapping clinical sections across EHRs to standardized sections.
• To leverage the eMERGE heart failure (HF) phenotyping algorithm for this assessment.

Main Methods:

• Utilized random forest and bidirectional encoder representations from transformers (BERT) models.
• Trained ML models on an automated labeled corpus from an HL7 CDA-compliant EHR system.
• Assessed performance using a blind test set and a manually annotated gold standard from multiple EHR systems.

Main Results:

• ML model performance (F-measure) varied significantly (0.00-0.91%), indicating limitations in a single tuning parameter set.
• Error analysis revealed that sections often deviate from standardized formats, impacting performance.
• Heterogeneity in section structure across EHRs directly affects the accuracy of ML-based phenotyping.

Conclusions:

• ML techniques show potential for mapping clinical sections across EHRs to standard formats.
• Poor adoption of documentation standards and EHR data quality significantly hinder the effectiveness of these ML applications.
• Addressing EHR data quality and standardization is essential for reliable computational phenotyping.