Related Experiment Video
Updated: Jan 10, 2026
06:04
Systematic Hearing Performance Evaluation Process for Adolescents with Cochlear Implantation at Early Ages
Published on: March 24, 2023
736
Assessing the Performance and Reliability of ChatGPT in Answering Patients' Questions on Voice Disorders across Time
1Independent Researcher in Laryngology, Voice Pathology, and Speech- Language Pathology, Tehran, Iran.
Journal of Voice : Official Journal of the Voice Foundation
|November 22, 2025
Summary
ChatGPT provides accurate voice disorder information but its readability is low for the public, though newer versions show improvement. Further research is needed before recommending it for patient medical information.
Area of Science:
- Artificial Intelligence in Healthcare
- Speech-Language Pathology
- Medical Information Dissemination
Background:
- Voice disorders affect a significant population, necessitating accessible and accurate information.
- Current information sources may not always meet patient needs for clarity and accessibility.
- Large language models (LLMs) like ChatGPT offer potential for patient education but require evaluation.
Purpose of the Study:
- To assess the accuracy, safety, and quality of ChatGPT's responses to common voice disorder questions.
- To evaluate the readability and empathy of AI-generated responses.
- To compare responses across different time slots to identify potential improvements in the AI model.
Main Methods:
- 30 common voice disorder questions were curated by a speech-language pathologist and online resources.
- Questions were input into GPT-4o mini in November 2024 and April 2025 using a "VoiceHelp" chatbot prompt.
- Responses were evaluated for accuracy, harm, consensus alignment, empathy, and overall quality.
- Readability was assessed using Flesch Reading Ease Score (FRES), Gunning Fog Scale Level (GFSL), and Dale-Chall Score (D-CS).
Main Results:
- Over 90% of responses were accurate and harmless, with 80% aligning with medical consensus.
- The majority of responses (92.5%) were rated as acceptable to very good quality, despite 38.3% lacking empathy.
- Readability scores indicated a high level of complexity for the general public, though improvements were noted in the later time slot.
- Response length and complexity increased significantly between the two time slots.
Conclusions:
- ChatGPT demonstrates accuracy and informativeness in answering voice disorder queries.
- Readability remains a challenge for general audiences, though newer versions show progress.
- Further investigation is required to establish ChatGPT as a trusted medical information source for patients with voice disorders.
