Related Experiment Videos

Guideline Concordance of Large Language Model Responses to Parent-oriented Guideline Prompts About Pediatric Acute

Ahmet Murat Çörekci1, Belen Ateş2, Orkun Dinç3

  • 1From the Department of Orthopaedics and Traumatology.

The Pediatric Infectious Disease Journal
|July 28, 2026
PubMed

Insights

Large language models (LLMs) demonstrate high concordance with pediatric arthritis guidelines. While useful for caregiver education, LLM information should not replace professional medical advice or clinical assessment.

Area of Science:

  • Artificial Intelligence in Medicine
  • Medical Informatics
  • Clinical Guidelines

Background:

  • Patients and caregivers increasingly use large language models (LLMs) for medical information.
  • Accurate information is crucial in pediatric acute bacterial arthritis due to the need for timely diagnosis and management.
  • This study assesses LLM response concordance with established clinical guidelines.

Purpose of the Study:

  • To evaluate the accuracy of LLM-generated medical information for pediatric acute bacterial arthritis.
  • To benchmark responses from different LLMs against Pediatric Infectious Diseases Society/Infectious Diseases Society of America (PIDS/IDSA) guidelines.
  • To assess the suitability of LLMs for caregiver education regarding pediatric arthritis.

Main Methods:

  • Benchmarking study using 27 PIDS/IDSA guideline recommendations reformulated into parent-oriented prompts.
  • Prompts submitted to GPT-5.4 Thinking, Gemini 3 Thinking, and Claude 4.6 Sonnet.
  • Responses anonymized, independently reviewed by blinded experts, and classified for concordance.

Main Results:

  • Overall concordance with PIDS/IDSA guidelines was high at 92.6% (75/81 responses).
  • Gemini 3 Thinking achieved 100% concordance (27/27), Claude 4.6 Sonnet 92.6% (25/27), and GPT-5.4 Thinking 85.2% (23/27).
  • No unsupported or hallucinated content was found; moderate interrater agreement (κ = 0.580).

Conclusions:

  • LLMs exhibit high guideline concordance for pediatric acute bacterial arthritis information.
  • Despite high concordance, LLMs may not perform independent clinical reasoning.
  • LLMs can aid caregiver education but must not substitute clinician judgment or guideline-based care.
Abstract

Keywords:
PIDS/IDSA guidelineacute bacterial arthritis in pediatricsartificial intelligenceguideline concordancelarge language modelsparent-oriented medical information

Related Concept Videos

Tonsillitis II: Management01:26

Tonsillitis II: Management

This lesson will focus on the different treatment options for managing tonsillitis, which typically depend on the cause and severity.
Acute Pharyngitis01:30

Acute Pharyngitis

Introduction
Acute pharyngitis is the inflammation of the back of the throat (pharynx), commonly resulting in a sore throat. It is a frequently encountered condition that prompts individuals to seek medical advice.
Classification
Acute pharyngitis can be categorized based on its underlying cause:
Streptococcal Pharyngitis01:27

Streptococcal Pharyngitis

Streptococcal pharyngitis, commonly known as “strep throat,” is an acute infection of the oropharyngeal tissues caused by the Gram‑positive Group A Streptococcus (Streptococcus pyogenes). Transmission occurs primarily through respiratory droplets expelled during coughing, sneezing, or talking.Mechanisms of Host Entry and Immune EvasionUpon entering the host, S. pyogenes adheres to the mucosal epithelial cells of the pharynx via surface proteins, notably lipoteichoic acid and the antiphagocytic...
Drug Dosing: Infants and Children01:29

Drug Dosing: Infants and Children

Pediatric patient dosages diverge from adults due to disparities in body surface area, total body water, and extracellular fluid per kilogram of body weight. The dosing regimen considers the variations in pharmacokinetics and pharmacology across distinct age groups, encompassing preterm newborns, infants, young children, older children, and adolescents. Calculation of pediatric patient doses is predicated on determining body surface area, which exhibits a superior correlation with the child's...
Tonsillitis I: Introduction01:30

Tonsillitis I: Introduction

Tonsillitis is inflammation of the tonsils, which are two lymphoid tissue masses at the back of the throat. This condition can cause discomfort and irritation in the throat.
Etiology
Three primary contributing factors have been identified.
SBAR II: Application of SBAR01:14

SBAR II: Application of SBAR

SBAR is an effective communication tool used by healthcare professionals to communicate patient information accurately. SBAR stands for Situation, Background, Assessment, and Recommendation. For a better understanding, an example is given below.
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...