Related Experiment Video
Updated: Jun 19, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Evaluation of Large Language Models in Infectious Disease Decision-Making: From Examination to Clinical Practice
Dandan Wu1, Keying Chen1, Xiaocui Wu1
1Department of Infectious Diseases, The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, Zhejiang, People's Republic of China.
Infection and Drug Resistance
|June 18, 2026
Summary
Large language models (LLMs) show promise in infectious disease clinical reasoning, performing comparably to residents on exams. Human-AI collaboration enhances decision quality but clinician oversight remains crucial for safety.
Area of Science:
- Artificial Intelligence in Medicine
- Clinical Decision Support Systems
- Infectious Disease Management
Background:
- The integration of artificial intelligence (AI), specifically large language models (LLMs), into clinical practice is rapidly evolving.
- Evaluating the efficacy of LLMs in specialized medical domains like infectious diseases is critical for understanding their potential and limitations.
Purpose of the Study:
- To assess the performance of leading large language models (LLMs) in infectious disease clinical reasoning and decision-making.
- To compare LLM capabilities against infectious disease residents using a dual assessment framework.
Main Methods:
- A comprehensive evaluation of four LLMs (DeepSeek-R1, ChatGPT-5, Grok 3, Gemini 2.5 Flash) was performed.
- A dual assessment framework included examination-based questions and real-world clinical case scenarios.
- LLM performance was benchmarked against infectious disease residents, with outcomes measured by accuracy, score rates, and expert review using Likert scales.
Main Results:
- LLMs demonstrated comparable performance to infectious disease residents in examination-based assessments (p=0.54).
- LLMs excelled in low-order, knowledge-based questions, while residents showed an advantage in simple case-based questions requiring higher-order reasoning.
- LLM-assisted clinical decision-making showed trends toward improved accuracy and significantly enhanced completeness compared to independent decisions, though high-risk errors highlight limitations.
Conclusions:
- Large language models (LLMs) offer significant potential for infectious disease education and clinical decision support, particularly for knowledge recall.
- Clinician oversight is essential due to LLM limitations in complex clinical reasoning.
- A collaborative human-AI approach maximizes decision quality and safety, necessitating refined regulatory frameworks for responsible clinical deployment.
Related Concept Videos
01:18Steps in Outbreak Investigation
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
01:26Principles of Disease Surveillance
Disease surveillance is the systematic collection, analysis, and interpretation of health data essential to the planning, implementation, and evaluation of public health practice. This process integrates data dissemination to entities responsible for preventing and controlling disease, injury, and disability. Surveillance systems provide crucial information for action, helping public health authorities make informed decisions to manage and prevent outbreaks, ensure public safety, optimize...
01:17Introduction to Language of Pathophysiology ll
This lesson explores key terms that describe how diseases progress, their outcomes, and their distribution in populations.Diagnostic tests identify diseases and monitor treatment. These include blood and urine tests, biopsies, imaging (X-ray, MRI), and detection of infectious agents.Remission is a reduction or disappearance of symptoms.Exacerbation refers to the worsening of symptoms, such as increased wheezing during an asthma attack.A precipitating factor triggers an acute episode, while a...
01:23Investigation of Disease Outbreaks
Multistate foodborne outbreaks pose significant public health risks and require meticulous investigation to identify sources and implement control measures. The Centers for Disease Control and Prevention (CDC) utilizes a dynamic seven-step process for these investigations, integrating data from laboratories, interviews, and environmental assessments to protect public health.Outbreak Detection: The detection of multistate outbreaks typically begins with PulseNet, the CDC's national laboratory...
01:28Infectious Diseases and Their Occurrence
Infectious diseases appear in populations through various transmission patterns, influenced by pathogen characteristics, population immunity, environmental conditions, and social behavior. Understanding these patterns is essential for effective public health surveillance and intervention. These categories—sporadic, outbreak, epidemic, pandemic, and endemic—help frame the nature and scope of disease events.Sporadic diseases occur irregularly and infrequently, without a predictable temporal or...
01:15Impact of Pharmacokinetic–Pharmacodynamic Models: Regulatory Decisions
PK–PD modeling has significantly influenced FDA regulatory decisions, particularly drug approval, dosage optimization, and labeling. These models integrate pharmacokinetics (PK) and pharmacodynamics (PD) to predict drug behavior and effects, aiding in optimizing dosing regimens and enhancing the probability of clinical trial success.One notable example is Nesiritide (Natrecor®), a recombinant human brain natriuretic peptide for treating acute decompensated congestive heart failure (CHF).