Related Experiment Video
Updated: Jun 27, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Large-language-models for pediatric diagnosis: Performance evaluation using real-world clinical notes from common and
Cristian Launes1,2,3, Paula Esteller-Cucala1, Marina Alvarez-Estape1
1Digital Strategy and Data Department Hospital Sant Joan de Déu Barcelona Spain.
Pediatric Investigation
|June 26, 2026
Summary
Advanced large language models (LLMs) show superior diagnostic accuracy in pediatrics, especially for rare diseases, compared to human clinicians. These AI tools offer promising potential for augmenting clinical decision-making in complex cases.
Area of Science:
- Artificial Intelligence in Medicine
- Pediatric Diagnostics
- Clinical Decision Support Systems
Background:
- Evaluation of large language models (LLMs) in pediatric diagnosis using real clinical cases is limited.
- Assessing LLM response consistency and rare disease recognition is crucial for clinical integration.
Purpose of the Study:
- To evaluate the diagnostic accuracy, consistency, and clinical usability of LLMs versus human clinicians in pediatric medicine.
- To compare LLM performance against real-world pediatric cases, including common and rare conditions.
Main Methods:
- A cross-sectional study compared four LLMs against 78 pediatric clinicians using 50 real clinical cases (25 rare, 25 common).
- Performance metrics included Top-1 and Top-5 diagnostic accuracy, response consistency, and qualitative clinician ratings.
- Extended clinical information was provided for 20 cases to assess diagnostic efficiency.
Main Results:
- Advanced LLMs, specifically o1-preview and Claude-3.5 Sonnet, significantly outperformed clinicians in diagnostic accuracy (Top-1: 60.0% and 59.0% vs. 48.2%).
- LLM performance advantages were most pronounced for rare diseases, with o1-preview showing a 6-fold higher Top-5 diagnostic odds.
- Human-AI complementarity analysis demonstrated a 10-percentage-point accuracy uplift (94.3% union accuracy with o1-preview).
Conclusions:
- Newer LLMs demonstrated superior performance over human clinicians in complex pediatric diagnostics, particularly for rare diseases.
- Findings support the evaluation of LLMs as augmentative diagnostic tools in pediatric settings.
- Further research is needed to establish appropriate legal, ethical, and clinical oversight frameworks for AI in healthcare.