Related Experiment Video
Updated: Aug 6, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Enabling interoperability across disparate health data sources using Large Language Models
Obinwa Ozonze1, Ogechukwu Okonor2, Uzondu Dike3
1School of Computing, University of Portsmouth, Portsmouth, Hampshire, United Kingdom.
PLOS Digital Health
|July 23, 2026
Summary
Large Language Models (LLMs) show promise for automating health data integration by mapping between Common Data Models (CDMs) and FHIR R4. While DeepSeek performed best, human oversight is crucial for reliable clinical and research applications.
Area of Science:
- Health Informatics
- Artificial Intelligence in Healthcare
- Data Interoperability
Background:
- Health data system heterogeneity hinders interoperability and data integration.
- Multiple Common Data Models (CDMs) complicate integration, requiring labor-intensive schema mapping.
- This complexity challenges clinical decision support and AI-driven research.
Purpose of the Study:
- To evaluate the efficacy of three leading Large Language Models (LLMs) in mapping data elements between widely used CDMs and FHIR R4.
- To assess the accuracy and consistency of ChatGPT, Gemini, and DeepSeek in automating health data integration tasks.
- To identify common error patterns and domains with highest error rates during LLM-based data mapping.
Main Methods:
- Evaluated ChatGPT, Gemini, and DeepSeek for matching data elements from PCORnet, OMOP, Sentinel, and i2b2 to FHIR R4.
- Utilized web-based chat interfaces and standardized prompts across 36 runs (3 LLMs x 4 schemas x 3 repetitions).
- Measured accuracy using F1-scores with 95% confidence intervals, strict (AND) and relaxed (OR) match criteria, benchmarking against HL7 guides.
Main Results:
- DeepSeek achieved the highest mean F1-scores (0.78-0.99) and strong consistency (AND Match = 0.87).
- Gemini demonstrated good coverage (OR Match = 0.96) but higher variability (AND Match = 0.64); ChatGPT showed moderate accuracy (AND Match = 0.51).
- Demographics and Condition domains mapped most accurately; Observation had the highest error rate (44%). Semantic confusion was the primary error type (42%).
Conclusions:
- LLMs offer a feasible, complementary approach to automate health data integration, particularly for well-structured CDMs.
- Human-in-the-loop validation, prompt optimization, and metadata integration are essential for safe and reliable deployment.
- Further research into error patterns and model refinement is needed to enhance clinical and research utility.
