[A retrieval-augmented generation-based clinical decision support system for the restoration of endodontically...

Objective: To develop and evaluate Clinical Decision Surpport System-augmented by retrieval-augmented generation (RAG)-Professional (CDSS-RAG-Pro), a RAG-based clinical decision support system, through multi-level comparison of multiple large language models (LLM), and to provide an objective decision-making aid for the clinical restoration of endodontically treated teeth. Methods: A structured knowledge base was constructed by systematically retrieving and incorporating authoritative prosthodontic sources, including the Guideline for the Restoration of Tooth Defects, Dentition Defects and Edentulism (2022 Edition). A RAG module was developed accordingly and integrated into three LLM (DeepSeek-V3, Qwen 2.5-72B-Instruct, and GPT-3.5-turbo) to form the CDSS-RAG-Pro clinical decision support system for post-endodontic restoration. To evaluate the decision-making capability of this system, an expert panel was established. Thirty typical clinical cases were selected from the Department of Prosthodontics, Hospital of Stomatology, Sun Yat-sen University, from May 2024 to December 2025. The three LLM generated restorative plans under both RAG and non-RAG modes. Using a Likert scale, the expert panel scored the restorative plans across three dimensions: decision accuracy, logical coherence, and professional readability. Meanwhile, the expert panel drafted standard restorative reports. Two objective metrics, F1 score of the recall-oriented understudy for gisting evaluation based on longest common subsequence (ROUGE-L F1) and F1 score of the bidirectional encoder representations from Transformer-based semantic similarity score (BERTScore F1), were used to evaluate the semantic similarity between the generated restorative plans and the standard restorative reports. ROUGE-L F1 focused on assessing the model' s mastery of textual knowledge such as restorative guidelines, while BERTScore F1 focused on semantic consistency in reasoning patterns and decision-making logic. Generalized estimating equations were used to analyze differences in decision performance across LLM and between RAG and non-RAG modes. Results: Compared with the non-RAG mode, all models showed improved performance across all evaluation metrics in the RAG mode, with statistically significant differences (all P0.05). The highest BERTScore F1 in the RAG mode was 0.91 (DeepSeek-V3). Significant differences were observed among the three LLM across all evaluation metrics (all P0.05). Significant interaction effects were found between model type and RAG mode for decision accuracy score (interaction χ²=10.11, P0.001) and ROUGE-L F1 (interaction χ²=8.57, P0.05). Conclusions: The RAG-based CDSS-RAG-Pro improves the accuracy and interpretability of general-purpose LLM in decision-making for post-endodontic tooth restoration. This study demonstrates the unique clinical safety and scalability advantages of RAG as a lightweight, small-sample-oriented LLM configuration strategy to empower prosthodontic clinical decision-making.