Assessing the Credibility of ChatGPT on Temporomandibular Disorders

Large Language Models such as ChatGPT are increasingly used in dental education; however, their credibility in clinical contexts remains uncertain. This study is aimed to analyse the credibility and efficacy of responses given by ChatGPT about temporomandibular disorders among dentists and dental students.

Nine questions related to TMDs were posed to ChatGPT 3.5, and its responses were used to create an online survey. A total of 115 participants (60 dental students and 55 dentists) rated each response on a five-point Likert scale. Additionally, a Delphi panel of 14 TMD specialists assessed the same responses for accuracy, completeness, and guideline adherence based on DC/TMD and AAOP criteria. Consensus was defined as ≥ 70% agreement among panellists. The Mann-Whitney U test was used. A significance level of p < 0.05 was considered.

The study indicates positive perceptions with median values for all questions scoring more than 4. Descriptive statistics, revealing mean scores ranging from 4.28 to 4.48. The score of each response did not show a significant difference between groups. Dental students demonstrated significantly higher total scores across all nine responses compared with the dentist group (p = 0.029). Delphi findings indicated strong expert consensus, with median scores ≥ 4 and interquartile ranges mostly equal to 1.

ChatGPT 3.5 produced accurate and coherent responses about TMDs but should be used as a supplementary educational tool under professional supervision. Incorporating Delphi-based expert validation strengthened objectivity and demonstrated the value of combining user feedback with expert consensus when assessing AI-generated medical information.