Curated Retrieval-Augmented Generation for Dental Traumatology

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Dental Traumatology

Background:

• General-purpose large language models (LLMs) show promise but require validation for specialized clinical domains.
• Dental traumatology involves complex decision-making with critical implications for patient outcomes.
• Existing LLMs may lack the accuracy and source traceability needed for reliable clinical application.

Purpose of the Study:

• To validate DT-RAG, a retrieval-augmented generation system, for dental traumatology decision support.
• To compare the performance of DT-RAG against leading commercial large language models.
• To assess the accuracy, rationale validity, and clinical utility of DT-RAG.

Main Methods:

• DT-RAG was developed using a knowledge base of 250 curated text units from authoritative dental traumatology sources.
• Study 1: 99 binary clinical questions were posed to DT-RAG and eight commercial LLMs, comparing modal accuracy.
• Study 2: Seven specialists evaluated DT-RAG and three frontier LLMs on ten clinical scenarios using a detailed rubric.

Main Results:

• DT-RAG achieved 96.0% modal accuracy, significantly outperforming the best commercial LLM (87.9%).
• The curated knowledge base improved the base model's valid rationale rate from 49.5% to 98.0%, eliminating confabulations.
• Specialists rated DT-RAG highest (89.3%), significantly exceeding other LLMs and ranking it first in all evaluations.

Conclusions:

• DT-RAG, a curated retrieval-augmented system, surpasses general-purpose LLMs in dental traumatology decision support.
• The system provides accurate, source-traceable, and reproducible responses, making errors auditable.
• This approach holds potential for other clinical domains with established guidelines, pending prospective safety evaluation.