Comparative Analysis of Generative AI Language Models in Orthodontics: Evidence-Based Insights Into Perplexity, iASK,...

Area of Science:

• Orthodontics
• Artificial Intelligence
• Medical Informatics

Background:

• Large language models (LLMs) are increasingly used in healthcare.
• Evaluating the scientific reliability of LLMs for clinical applications is essential.
• Orthodontics presents unique challenges for AI-driven information retrieval.

Purpose of the Study:

• To compare the scientific reliability of three LLMs: Perplexity, iASK, and ChatGPT 4o mini.
• To assess the performance of these AI models in responding to clinical orthodontic questions.
• To determine the suitability of LLMs for orthodontic information needs.

Main Methods:

• Ten clinical orthodontic questions were posed to each LLM.
• Responses were independently scored by two evaluators using a 0-10 scale.
• Statistical analyses included correlation and reliability tests (Cronbach's alpha, Wilcoxon signed-rank test).

Main Results:

• Perplexity achieved the highest mean score (7.2), outperforming iASK (5.4) and ChatGPT 4o mini (5.2).
• A significant difference in performance was observed between Perplexity and the other two models (p = 0.002).
• High inter-evaluator reliability was confirmed (Cronbach's alpha = 0.947; Pearson's r = 0.982).

Conclusions:

• Perplexity demonstrated superior scientific reliability for orthodontic queries compared to iASK and ChatGPT 4o mini.
• The study underscores the need for rigorous evaluation of AI models before clinical implementation.
• Findings suggest potential for AI in orthodontics, pending further validation.