Comparative Evaluation of Diagnosis and Treatment Plan Given by Pediatric Dentists and Generated by ChatGPT: A...

Artificial intelligence (AI) is rapidly transforming healthcare, including dental specialties such as pediatric dentistry. Chat generative pre-trained transformer (ChatGPT), an AI language model, has demonstrated potential in patient communication and academic writing, but its clinical utility in diagnostic and treatment planning contexts remains underexplored.

To evaluate the diagnostic accuracy and treatment planning capabilities of ChatGPT in comparison with those of experienced pediatric dentists.

A cross-sectional pilot study was conducted involving 10 children aged 6-12 years. Standardized case scenarios were developed using clinical findings, medical history, and radiographs. Four pediatric dentists independently assessed each case and provided diagnoses and treatment plans. The same case scenarios were input into ChatGPT (v3.5), and its outputs were reviewed by a fifth pediatric dentist. A senior pediatric dentist blinded to the source validated all responses. Accuracy was compared, and inter-rater agreement was assessed using Cohen's kappa statistics.

ChatGPT achieved an 80% accuracy rate in diagnosis and treatment planning, equivalent to two pediatric dentists and slightly lower than the 90% accuracy seen in the other two. Statistical analysis revealed no significant difference between the performance of ChatGPT and the pediatric dentists (p = 0.926). Cohen's kappa indicated almost perfect agreement with two dentists (κ = 1.00) and moderate agreement with the others (κ = 0.615).

ChatGPT demonstrated diagnostic accuracy comparable to pediatric dentists and may serve as a valuable adjunct in clinical pediatric dentistry. Larger studies are needed to confirm these findings and guide responsible integration of AI tools.