Orthodontic Biomechanical Reasoning with Multimodal Language Models: Performance and Clinical Utility

Area of Science:

• Artificial Intelligence in Dentistry
• Orthodontic Biomechanics
• Clinical Decision Support Systems

Background:

• Multimodal large language models (LLMs) are emerging as clinical support tools.
• Their capability for orthodontic biomechanical reasoning remains unevaluated.
• This study assesses LLMs' ability to analyze orthodontic treatment mechanics.

Purpose of the Study:

• To systematically evaluate the capacity of multimodal large language models (LLMs) for orthodontic biomechanical reasoning.
• To explore the potential role of LLMs in supporting orthodontic decision-making.
• To compare the performance of different LLMs in analyzing orthodontic force systems.

Main Methods:

• Five multimodal large language models (LLMs) analyzed 56 standardized intraoral photographs of orthodontic force systems.
• Three experienced orthodontists evaluated model outputs on observation, interpretation, biomechanics, and confidence using a 5-point scale.
• Inter-rater reliability and statistical comparisons between models were performed.

Main Results:

• GPT-o3 achieved the highest composite score (66.8%), significantly outperforming Claude (57.8%), Gemini (52.6%), GPT-4.0 (48.8%), and Grok (38.8%).
• Excellent inter-rater reliability was observed among expert evaluators (ICC: 0.786-0.802).
• Model self-reported confidence poorly correlated with expert-rated output quality.

Conclusions:

• Multimodal large language models (LLMs) demonstrate emerging potential in assisting orthodontic biomechanical assessment.
• Expert-guided validation can enhance LLM contributions to clinical decision support in orthodontics.
• LLMs may offer valuable support across various biomechanical scenarios in routine orthodontic care.