AI-Powered Clinical Decision Support in Dentistry: Comparative Evaluation of Large Language Models for Oral Medicine...

BACKGROUND This study evaluates the diagnostic performance of 3 prominent artificial intelligence (AI)-powered large language models (LLMs) - ChatGPT, Copilot, and Gemini - as AI assistants for the diagnosis of oral lesions and periodontal conditions using comprehensive statistical analysis. MATERIAL AND METHODS A retrograde study was conducted on 385 cases with definite diagnoses from the College of Dentistry, Taibah University, Saudi Arabia. Clinical and radiographic images were presented to each AI model, and the diagnostic performance of the LLMs was evaluated using a 5-point Likert scale across 8 criteria: diagnostic concordance, time efficiency, ease of use, clarity of explanation, comprehensiveness, ability to answer questions, reliability, and diagnostic range. Statistical analysis included descriptive statistics with 95% confidence intervals, Friedman tests, post-hoc pairwise comparisons, correlation analysis, effect size calculations, and reliability assessment using Cronbach's alpha. RESULTS ChatGPT demonstrated superior performance with an overall score of 4.846±0.075, followed by Copilot (4.433±0.163) and Gemini (4.234±0.088). Friedman tests revealed statistically significant differences across all evaluation criteria (P<0.001). Post-hoc analyses showed ChatGPT significantly outperformed both Gemini and Copilot in all criteria. Internal consistency was excellent for all systems (Cronbach alpha: 0.801-0.911). CONCLUSIONS The LLMs, particularly ChatGPT, demonstrate significant potential as reliable AI assistants for oral and periodontal diagnosis. The comprehensive statistical analysis confirms the superior performance of ChatGPT across multiple evaluation dimensions, supporting its potential integration into clinical practice.