Assessment of AI-Driven Large Language Models for Orthodontic Aesthetic Scoring Using the IOTN-AC

Area of Science:

• Dentistry
• Artificial Intelligence
• Medical Informatics

Background:

• The Aesthetic Component of the Index of Orthodontic Treatment Need (IOTN-AC) is crucial for determining orthodontic treatment necessity.
• Evaluating the accuracy of AI-based large language models (LLMs) in aesthetic assessments is essential for potential clinical integration.

Purpose of the Study:

• To assess the accuracy of AI LLMs (ChatGPT-5 and ChatGPT-5 Pro) in evaluating the IOTN-AC.
• To compare the performance of different AI models against experienced clinician assessments.

Main Methods:

• 150 intraoral photographs were scored by two AI LLMs and two expert clinicians.
• AI model performance was evaluated using IOTN-AC scores and treatment need classifications.
• Statistical analyses included correlation, Kappa, ICC, MAE, and Bland-Altman analysis.

Main Results:

• Both AI models showed significant correlations with expert scores (p < 0.001).
• ChatGPT-5 outperformed ChatGPT-5 Pro with lower Mean Absolute Error (MAE) and higher classification accuracy (66.7%).
• While generally unbiased, AI models did not consistently achieve high performance across all evaluation parameters.

Conclusions:

• AI LLMs demonstrate potential for orthodontic aesthetic assessment but require further development.
• Consistent high performance across all metrics is necessary before safe clinical adoption of AI in orthodontics.
• Statistical accuracy alone is not sufficient for the clinical application of AI in evaluating orthodontic treatment needs.