Related Experiment Video
Updated: Sep 12, 2025
A Computer-Based Platform for Aiding Clinicians in Eating Disorder Analysis and Diagnosis
Published on: May 10, 2022
ChatGPT Responses to Questions About Clinical Decision Making: A Preliminary Analysis of Accuracy and Completeness
Vijayachandra Ramachandra1, Cody W Dew2, Dawna Duff2
1Department of Communication Sciences and Disorders, Marywood University, Scranton, PA.
Purpose:
The primary aim of this preliminary study was to examine the potential advantages and disadvantages of using ChatGPT-3.5 (free version) to obtain information pertaining to the discipline of communication sciences and disorders (CSD).
Method:
Nine university-based CSD researchers with expertise in different topic areas created questions of varying complexity (basic, advanced, evidence-based practice [EBP], and general CSD) to ask ChatGPT and then rated the software's responses for accuracy (6-point Likert scale) and completeness (3-point Likert scale).
Results:
Across all questions (n = 162), the median accuracy was 5 (nearly all correct), with an M of 5.01. Regarding completeness, the median was 3 (comprehensive), and the mean was 2.41. Furthermore, there were no significant differences in composite ratings (accuracy and completeness combined) between responses to basic, advanced, and EBP questions; however, responses to general CSD questions received significantly higher composite ratings than responses to basic, advanced, and EBP questions. Despite strong performance of ChatGPT on average, the distribution of accuracy and completeness ratings showed that more than half of ChatGPT's responses were rated as not fully accurate, only 45.68% of the responses received a rating of 6 (correct). On completeness ratings, only 54.32% of responses received a rating of 3 (comprehensive).
Conclusions:
Our results indicate that ChatGPT can quickly provide answers and often generate responses that experts judge as accurate. However, clinicians cannot depend on ChatGPT for accurate and complete responses, especially for complex clinical questions. Using ChatGPT for clinical decision making may be particularly risky for inexperienced or student clinicians, who may have difficulty spotting inaccuracies.
Supplemental Material:
https://doi.org/10.23641/asha.29565167.
More Related Videos
05:04Author Spotlight: Evaluating Clinicians' Adoption of Ultrasound-Guided Vascular Cannulation Through Simulation Training
Published on: August 9, 2024
13:44Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
Related Concept Videos
01:25Critical Thinking II
01:09Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
01:13Patient-centered Care
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:21Methods of Documentation III: PIE
01:16Clinical Trials
There are four phases in a clinical trial. A phase one...