Related Experiment Video
Updated: Jan 7, 2026
Author Spotlight: 3D Movement Assessment of Maxillary Posterior Teeth in Clear Aligner Treatment
Published on: February 23, 2024
Performance of five free large language models in dental trauma: a 30-day longitudinal benchmark study
Rafaela Mancini Lisboa1, Arian Braido2, Adriana de-Jesus-Soares2
1Departament of Dentistry, Centro Universitário das Faculdades Associadas de Ensino - UNIFAE, São João da Boa Vista, Brazil.
Objective:
To compare the accuracy and consistency of five large language models (LLMs) in generating responses about dental trauma.
Materials And Methods:
Sixty dichotomous (true/false) questions were submitted daily to each LLM (ChatGPT, Google Gemini, Microsoft Copilot, DeepSeek, and Meta AI) for 30 days, totaling 18,000 responses. All interactions were performed under two prompting conditions (zero-shot and zero-shot with context). LLM responses were compared against the International Association of Dental Traumatology (IADT) guidelines. Statistical analysis was conducted using a generalized linear mixed model (GLMM) with a binomial distribution (α = 0.05), alongside calculation of sensitivity, specificity, accuracy, and area under the ROC curve (AUC) based on the 60-item set. Temporal stability was assessed using the intraclass correlation coefficient ICC.
Results:
All LLMs achieved accuracy above 85%, with Microsoft Copilot (91.1%) and DeepSeek (90%) performing best; no significant difference was observed between them (p > 0.05), but both outperformed the other models (p < 0.05). DeepSeek and Microsoft Copilot also showed the highest consistency over 30 days (ICC > 0.90).
Conclusion:
All evaluated LLMs, particularly Copilot and DeepSeek, demonstrated high accuracy in providing information on dental trauma, with stable performance over time. While the use of a context prompt did not significantly affect accuracy or stability.
