Evaluating retrieval-augmented generation for guideline-grounded textual planning in implant dentistry: A comparative...

Area of Science:

• Artificial Intelligence in Dentistry
• Biomedical Informatics
• Clinical Decision Support Systems

Background:

• Large language models (LLMs) show promise in clinical applications but often lack citation precision and reliability.
• Retrieval-augmented generation (RAG) frameworks aim to enhance LLM performance by integrating external knowledge bases.

Purpose of the Study:

• To evaluate if a RAG framework improves citation precision and clinical reliability of LLMs in implant dentistry.
• To compare the performance of a standard LLM against a RAG-LLM using clinical vignettes.

Main Methods:

• A domain-specific knowledge base was created from consensus reports and treatment guides.
• Standard and RAG-LLMs processed 40 clinical vignettes, generating 160 treatment plans.
• Two experts evaluated plans in a double-blinded, consensus-driven manner.

Main Results:

• RAG-LLMs significantly improved evidence traceability and citation precision (P=0.046).
• Standard LLMs showed superior biomechanical spatial planning (P=0.025).
• RAG exhibited retrieval bias in soft-tissue cases, leading to inappropriate hard-tissue augmentation.

Conclusions:

• RAG enhances evidence traceability and citation precision in implant dentistry AI.
• Retrieval bias in RAG can lead to overtreatment, necessitating expert oversight.
• RAG-LLMs are valuable adjunctive tools but cannot replace clinician spatial reasoning.