Related Experiment Video
Updated: Apr 7, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Comparative Impact of ChatGPT and Conventional Search Tools on Clinical Reasoning Performance: A Randomized Crossover
Adisak Nartthanarung1, Komson Plangsiri2, Pinkawas Kongmalai1
1Department of Orthopedics, Faculty of Medicine, Kasetsart University, Bangkok, Thailand.
Purpose:
To compare the impact of ChatGPT and conventional search strategies on clinical reasoning performance among preclinical medical students.
Patients And Methods:
A randomized crossover study was conducted at a single institution during a musculoskeletal system course involving 46 second-year medical students. Participants completed a baseline pre-test followed by two structured intervention phases in which they analyzed standardized clinical cases using either ChatGPT or conventional search tools (Google or PubMed). A 60-minute washout period was implemented before crossover to the alternate modality. Post-tests were administered after each phase. The primary outcome was clinical reasoning performance measured using an eight-point rubric-based scale. Secondary outcomes included self-perceived learning, confidence, and qualitative feedback. Paired t-tests were used for within-subject comparisons, and effect sizes were calculated using Cohen's d.
Results:
The mean pre-test score was 3.96 (standard deviation 1.65), increasing to 4.96 (standard deviation 1.71) after the first intervention and to 5.70 (standard deviation 1.50) after crossover. Improvements were statistically significant across all paired comparisons (p < 0.05), with a large cumulative effect size (Cohen's d = 0.97). Performance improvements were observed across both learning modalities, without evidence that gains were attributable to a single approach. Students reported that ChatGPT facilitated rapid organization of differential diagnoses and management plans, whereas conventional search encouraged more deliberate synthesis and comparison of information sources.
Conclusion:
Both artificial intelligence-assisted and conventional search strategies improved short-term clinical reasoning performance within a structured active-learning environment. These findings support a balanced integration of large language models alongside traditional search methods in undergraduate medical education.
Clinical Trial Registration Number:
TCTR20260218005.
Related Concept Videos
Patient-centered Care
Critical Thinking II
Clinical Trials
There are four phases in a clinical trial. A phase one...
Methods of Documentation III: PIE
Clinical Trials: Overview
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
