Using Prompt Engineering to Optimize a RAG Pipeline for EHR-Nursing Data Standardization

Area of Science:

• Health Informatics
• Natural Language Processing
• Nursing Informatics

Background:

• Electronic health records (EHRs) contain valuable nursing care plan data.
• Heterogeneity in local terminologies hinders EHR data standardization and interoperability.
• Standardized nursing data is crucial for large-scale research and clinical decision-making.

Purpose of the Study:

• To evaluate an optimized Retrieval-Augmented Generation (RAG) pipeline for mapping local nursing care plan data to standardized terminologies.
• To assess the effectiveness of different prompt engineering techniques within the RAG pipeline.
• To determine the feasibility of using large language models for semantic interoperability in nursing informatics.

Main Methods:

• Developed and optimized a RAG pipeline integrating large language models.
• Applied the pipeline to map nursing care plan problems, goals, and interventions from two distinct health systems.
• Compared performance across various prompt engineering strategies, including structured prompts and few-shot learning.

Main Results:

• The optimized RAG pipeline demonstrated moderate accuracy (45-70%) in mapping local nursing terminologies to standardized ones.
• Structured Prompts combined with Few-Shot Learning yielded the highest performance among the tested techniques.
• The approach significantly reduced human workload compared to manual mapping.

Conclusions:

• Optimized RAG pipelines show promise for improving semantic interoperability of nursing data from EHRs.
• While full automation is not yet achieved, this human-in-the-loop approach supports scalable data standardization.
• Further refinement of prompt engineering and RAG models can enhance accuracy and efficiency for nursing informatics research.