Related Experiment Video
Updated: Jan 16, 2026
Author Spotlight: 3D Scanning and Augmented Reality for Enhanced Cancer Surgery Communication
Published on: December 15, 2023
Accuracy of ChatGPT-4 Plus in Providing Information on Oral Cancer Management
Busra Yilmaz1,2, Emine Nur Kahraman1, Michael T Brennan3,4
1Department of Oral Medicine and Maxillofacial Radiology, School of Dental Medicine, Istinye University, Istanbul, Turkey.
Objective:
Artificial intelligence (AI)-driven large language models, such as Chat Generative Pre-Trained Transformer (ChatGPT)-4 Plus, are increasingly used for patient education and clinical decision support in oral oncology, although their accuracy in oral cancer (OC) management remains uncertain. This study evaluates the accuracy of ChatGPT-4 Plus responses to clinically relevant questions regarding OC diagnosis, treatment, recovery, and prevention.
Methods:
A cross-sectional study assessed 65 clinically relevant OC-related questions using a paid ChatGPT-4 Plus subscription without modifications. Three oral medicine specialists and one radiation oncologist rated accuracy on a four-point scoring system. Interrater reliability was measured with the intraclass correlation coefficient (ICC), and chi-square tests were used for comparisons.
Results:
Among 65 questions, 63% of responses were Score 1, with none rated as Score 4. Score 1 was most frequent in Recovery (72%), followed by Treatment (62%), Prevention (60%), and Diagnosis (55%). Scores 2 and 3 responses were highest in Diagnosis (45%). Recovery had significantly higher Score 1 responses than Diagnosis (p < 0.05), while other comparisons were not significant. ICC ranged from 0.85 to 0.93.
Conclusions:
ChatGPT-4 Plus provided accurate responses to clinically relevant OC-related questions, particularly regarding recovery. However, diagnostic inconsistencies highlight the need for clinician oversight before integrating AI into practice.
