Nursing Retrieval-Augmented Generation: Retrieval augmented generation for nursing question answering with large language models | JoVE Visualize

Area of Science:

Artificial Intelligence in Nursing
Clinical Informatics
Natural Language Processing

Background:

Large language models (LLMs) show promise for healthcare applications but require domain-specific refinement for accuracy.
Ensuring the reliability and clinical applicability of AI-generated nursing information is crucial for patient safety.
Existing LLM-based systems may struggle with the nuanced and evidence-based requirements of nursing knowledge.

Purpose of the Study:

To develop and evaluate a Nursing Retrieval-Augmented Generation (NurRAG) system for accurate nursing question answering.
To assess the clinical applicability and performance of the NurRAG system compared to conventional LLMs.
To enhance the generation of evidence-based and guideline-concordant nursing responses.

Main Methods:

A multidisciplinary team designed the NurRAG framework, incorporating a nursing knowledge base, question filtering, semantic retrieval, and evidence-conditioned generation.
The system utilized document normalization, embedding, vector indexing, supervised classification, and semantic re-ranking for evidence selection.
Performance was evaluated using 1,000 expert-verified nursing Q&A pairs, measuring semantic fidelity (ROUGE-L) and clinical accuracy.

Main Results:

The NurRAG system significantly improved ROUGE-L scores and accuracy for both ChatGLM2-6B and LLaMA2-7B models compared to baseline LLMs (P < 0.001).
Accuracy increased from 49.08% to 75.83% for ChatGLM2-6B and from 43.27% to 73.29% for LLaMA2-7B.
Case analysis confirmed NurRAG's effectiveness in reducing hallucinations and producing evidence-based, guideline-concordant nursing answers.

Conclusions:

The NurRAG system effectively integrates domain-specific retrieval with LLM generation for accurate, reliable, and traceable nursing answers.
The findings support the feasibility of NurRAG to enhance clinical knowledge access and evidence-based nursing decision-making.
This AI approach holds potential for the safe and effective application of artificial intelligence in nursing practice.