Related Experiment Video
Updated: Mar 3, 2026
Studying Orthodontic Tooth Movement in Mice
Published on: August 2, 2024
'OrthoQA' semantic search and question-answering tool for orthodontic education
Sarah Abu Arqub1, Qian Zhao2, Prapti Akolkar2
1Department of Orthodontics, College of Dentistry, University of Florida, 1395 Center Drive, P.O. Box 100444, Gainesville, FL 32610-0444, United States of America.
Objective:
To develop and systematically evaluate a domain-specific retrieval-augmented question-answering system, 'OrthoQA', for orthodontic education.
Methods:
Using a retrieval-augmented generation (RAG) architecture, content from 18 orthodontic textbooks was integrated into 'OrthoQA' via a structured knowledge base processed through the University of Florida's NaviGator artificial intelligence (AI) Toolkit and HiPerGator infrastructure. The web interface, built with React.js and Flask, enables real-time responses with clickable citations. Evaluation included three phases: (1) automated benchmarking of seven LLMs, (2) comparison of RAG versus non-RAG configurations, and (3) expert evaluations. Pairwise comparisons were analyzed using the Wilcoxon signed-rank test.
Results:
Automated analysis identified GPT-4o and Claude-3.5-Sonnet as top-performing models, while LLaMA-3.1-70B-Instruct was the strongest open-source alternative. RAG integration improved factual completeness and contextual precision without compromising clarity or topical relevance. In expert evaluations of 50 orthodontic questions, 'OrthoQA' with its designated base model demonstrated comparable mean ratings of accuracy (2.82-4.56), completeness (1.40-2.02), and clarity (2.48-2.80) to ChatGPT-4o (accuracy 4.26-4.62; completeness 2.02-2.14; clarity 2.80-2.84) and LLaMA-3.1-70B-Instruct (accuracy 3.22-3.36; completeness 1.44-1.64; clarity: 2.32-2.46). Both evaluators reported no significant differences in preference.
Conclusions:
By combining retrieval augmentation with authoritative sources, 'OrthoQA' provides contextually grounded answers in educational settings where access to general-purpose LLMs may be restricted due to institutional policies or data privacy concerns, thereby representing a safe and reliable alternative question-answering system.
Clinical Significance:
OrthoQA' supports evidence-based orthodontic education by delivering accurate, citation-backed responses within secure institutional environments, facilitating standardized learning without reliance on unrestricted general-purpose AI platforms.
