Related Experiment Video
Updated: Sep 18, 2025
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Artificial intelligence for solving pediatric clinical cases: A Retrieval-Augmented approach utilizing Llama3.2 and
Gianluca Mondillo1, Simone Colosimo1, Alessandra Perrotta1
1Department of Woman, Child and of General and Specialized Surgery, Università degli Studi della Campania "Luigi Vanvitelli", Via Luigi De Crecchio 2, Naples, Italy.
Background:
The "hallucinations" of Large Language Models (LLMs) raise concerns about their accuracy in pediatrics. This study aimed to evaluate whether integrating information from the Nelson Textbook of Pediatrics through a Retrieval-Augmented Generation (RAG) system could enhance the performance of Llama3.2 in addressing complex pediatric clinical cases.
Methods:
We assessed the RAG system performance using 1,713 multiple-choice pediatric clinical questions from the MedQA dataset (n = 1,572) and Archives of Disease in Childhood-Education and Practice (n = 141). Each question was presented to Llama3.2 both in standalone mode and with RAG integration. The percentage of correct answers between models was compared using the chi-square test. p < 0.05 was considered statistically significant.
Results:
The RAG-integrated system significantly outperformed standalone Llama3.2, achieving an overall accuracy of 67.78 % (1,161/1,713) compared to 46.18 % (791/1,713) for Llama3.2 alone (p = 1.5e-112). The improvement was consistent across all pediatric subspecialties.
Conclusions:
Incorporating RAG systems into clinical decision-making can enhance reliability and safety.
More Related Videos
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
Related Concept Videos
01:27Nursing Clinical Information System
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
01:12Retrieval
Recall involves accessing information without cues, such as during an essay test, where individuals must retrieve facts and concepts from memory unaided. Another example is remembering the name of a colleague...