ArcMAP - ML assisted medical concept mapping to accelerate NHS data standardization

Area of Science:

• Health Informatics
• Biomedical Data Science
• Clinical Data Standardization

Background:

• Electronic health records (EHRs) are increasingly used for real-world evidence (RWE) studies.
• Heterogeneity in EHR data collection and local coding schemes hinders RWE study standardization.
• Manual medical concept mapping is labor-intensive and requires clinical expert review.

Purpose of the Study:

• To present ArcMAP, an end-to-end application for streamlining and accelerating medical concept mapping.
• To integrate a state-of-the-art biomedical representation model (BioLORD) into a human-in-the-loop workflow.
• To evaluate ArcMAP's performance in real-world deployment scenarios.

Main Methods:

• Developed ArcMAP, an application integrating the BioLORD model with a human-in-the-loop workflow.
• Implemented a continuous learning pipeline to capture expert feedback and update the model.
• Conducted comprehensive evaluations across multiple scenarios, including continuous fine-tuning and new hospital onboarding.

Main Results:

• Domain-specific fine-tuning significantly improved top-1 accuracy for laboratory test names from 37.0% to 91.6%.
• Simulated onboarding of a new hospital yielded a weighted average top-1 accuracy of 73.5%, highlighting system variability.
• Real-world use demonstrated increased mapping efficiency compared to manual workflows, with observed variations across sessions.

Conclusions:

• ArcMAP effectively streamlines medical concept mapping, enhancing efficiency in RWE studies.
• Continuous learning and domain adaptation are crucial for optimizing performance.
• Variability in data and coding practices across healthcare systems presents ongoing challenges for automated mapping solutions.