Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Language and Cognition01:27

Language and Cognition

345
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
345
Patient-centered Care01:13

Patient-centered Care

2.0K
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
2.0K
Critical Thinking II01:25

Critical Thinking II

2.8K
Critical thinking is a cognitive process with several attributes. The attributes of critical thinking include the following:
2.8K
Reasoning01:30

Reasoning

77
Reasoning is the action of thinking about something in a logical, sensible way. It is integral to problem-solving, decision-making, and critical thinking. Reasoning can be inductive or deductive. Reasoning involves transforming information into conclusions, which is essential for problem-solving, decision-making, and critical thinking.
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
77
Critical Thinking I01:24

Critical Thinking I

3.3K
Critical thinking helps decision-making and allows nurses to recognize barriers to success and find solutions to possible issues. It helps to brainstorm and implement ideas to achieve goals. Critical thinking helps acknowledge and state workflow inefficiencies while improving management techniques. Nurses understand the value of critical thinking and look for fellow nurses with critical thinking skills to upgrade their professional standards. Critical thinking can advance a nurse's career...
3.3K
Inductive Reasoning00:59

Inductive Reasoning

60.4K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
60.4K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Towards Conversational AI for Disease Management.

Nature·2026
Same author

Advancing conversational diagnostic AI with multimodal reasoning.

Nature medicine·2026
Same author

Severe allergic and anaphylactic transfusion reactions in consecutive recipients from the same donor.

Haematologica·2025
Same author

NetStart 2.0: prediction of eukaryotic translation initiation sites using a protein language model.

BMC bioinformatics·2025
Same author

Predicting the subcellular location of prokaryotic proteins with DeepLocPro.

Bioinformatics (Oxford, England)·2024
Same author

Physician Level Assessment of Hirsute Women and of Their Eligibility for Laser Treatment With Deep Learning.

Lasers in surgery and medicine·2024

Related Experiment Video

Updated: Jun 30, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

555

Can large language models reason about medical questions?

Valentin Liévin1,2, Christoffer Egeberg Hother3, Andreas Geert Motzfeldt1

  • 1Section for Cognitive Systems, Technical University of Denmark, Anker Engelunds Vej 101, 2800 Kongens Lyngby, Denmark.

Patterns (New York, N.Y.)
|March 15, 2024
PubMed
Summary

Large language models show promise in medical question answering, with GPT-3.5 and Llama 2 achieving passing scores on challenging US Medical Licensing Examination benchmarks. These AI models demonstrate reasoning and knowledge recall capabilities in complex medical scenarios.

Keywords:
GPT-3.5Llama 2MedQAlarge language modelsmachine learningmedicalopen sourceprompt engineeringquestion answeringuncertainty quantification

More Related Videos

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.5K
A Knowledge Graph Approach to Elucidate the Role of Organellar Pathways in Disease via Biomedical Reports
07:35

A Knowledge Graph Approach to Elucidate the Role of Organellar Pathways in Disease via Biomedical Reports

Published on: October 13, 2023

1.6K

Related Experiment Videos

Last Updated: Jun 30, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

555
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
05:56

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Published on: April 14, 2023

2.5K
A Knowledge Graph Approach to Elucidate the Role of Organellar Pathways in Disease via Biomedical Reports
07:35

A Knowledge Graph Approach to Elucidate the Role of Organellar Pathways in Disease via Biomedical Reports

Published on: October 13, 2023

1.6K

Area of Science:

  • Artificial Intelligence
  • Medical Informatics
  • Natural Language Processing

Background:

  • Large language models (LLMs) demonstrate advanced capabilities but their real-world performance in expert domains like medicine is not well understood.
  • Evaluating LLMs requires assessing their reasoning skills and domain-specific knowledge application.
  • Medical benchmarks provide a rigorous testbed for AI performance in healthcare contexts.

Purpose of the Study:

  • To investigate the performance of closed- and open-source LLMs on complex, real-world medical questions.
  • To assess the effectiveness of different prompting strategies, including chain-of-thought, few-shot, and retrieval augmentation.
  • To determine if LLMs can achieve expert-level performance on medical licensing examinations.

Main Methods:

  • Evaluation of GPT-3.5 and Llama 2 models on three medical question-answering datasets: MedQA-USMLE, MedMCQA, and PubMedQA.
  • Application of various prompting techniques: chain-of-thought (CoT), few-shot learning, and retrieval augmentation.
  • Expert annotation of generated CoTs to assess reasoning quality.
  • Utilizing prompt engineering techniques like few-shot and ensemble methods for GPT-3.5.

Main Results:

  • InstructGPT demonstrated proficiency in reading, reasoning, and recalling expert medical knowledge, as confirmed by expert annotation of its chain-of-thought outputs.
  • GPT-3.5, using advanced prompt engineering, achieved passing scores on MedQA-USMLE (60.2%), MedMCQA (62.7%), and PubMedQA (78.2%), with calibrated predictive distributions.
  • Open-source models are rapidly improving, with Llama 2 70B surpassing the MedQA-USMLE passing threshold (62.5% accuracy).

Conclusions:

  • LLMs, particularly GPT-3.5, show significant potential for assisting with complex medical reasoning and question answering.
  • Prompt engineering techniques are crucial for optimizing LLM performance in specialized domains.
  • The performance gap between closed- and open-source models in medical AI is narrowing, indicating progress in accessible AI solutions for healthcare.