DGADS: A graph-based agentic decision support system for precision dental question answering

Area of Science:

• Artificial Intelligence in Dentistry
• Medical Informatics
• Knowledge Representation

Background:

• Large language models (LLMs) show promise for dental applications but suffer from hallucinations.
• Accurate and reliable information retrieval is crucial for dental decision-making.
• Existing retrieval-augmented generation (RAG) methods need enhancement for specialized domains like dentistry.

Purpose of the Study:

• To develop and evaluate a graph-based agentic decision support system (DGADS) for precise dental question answering.
• To mitigate the hallucination problem in LLMs within the dental field.
• To enhance the reliability and traceability of information provided by AI systems in dentistry.

Main Methods:

• Developed a Dental Graph-based Agentic Decision Support System (DGADS) with knowledge graph builder, graph-based RAG, and agentic RAG modules.
• Constructed DentalKG, a knowledge graph with 130,735 entities and 236,935 triples, from dental textual data.
• Evaluated DGADS on internal MCQs, external MCQs, and open-ended questions, comparing against state-of-the-art LLMs and chunk-based RAG.

Main Results:

• DGADS achieved absolute accuracy improvements of 0.04 on internal and 0.05 on external multiple-choice questions.
• Performance on open-ended questions improved, with mean scores increasing up to 0.78 on a 5-point scale.
• DGADS demonstrated the ability to assess information sufficiency and retrieve external evidence when necessary.

Conclusions:

• The Dental Graph-based Agentic Decision Support System (DGADS) shows potential as a research tool for dental question answering.
• DGADS enhances precision and efficiency by grounding LLM outputs in a knowledge graph and retrieving external evidence.
• This system may improve clinical applications by reducing hallucinations and providing traceable rationale for dental information.