JADE: jawbone lesion diagnosis and decision supporting system

Area of Science:

• Dentomaxillofacial Radiology
• Artificial Intelligence in Medicine
• Diagnostic Assistance Systems

Background:

• Large language models (LLMs) show potential in medical diagnostics but require enhanced reasoning capabilities.
• Jawbone lesion assessment presents complex diagnostic challenges.
• Current diagnostic systems may lack the adaptability and comprehensive reasoning of advanced AI.

Purpose of the Study:

• To develop and evaluate JADE, a novel retrieval-augmented generation (RAG) system for diagnostic assistance in jawbone lesion assessment.
• To compare the diagnostic accuracy and stability of JADE against standalone LLMs and a supervised learning system (ORAD).
• To assess the impact of RAG on LLM performance in a specialized radiological domain.

Main Methods:

• JADE was built as a cloud-based RAG system, integrating an oral radiology database with LLM backbones.
• Structured clinical data was used for hybrid semantic and keyword-based retrieval to augment LLM prompts.
• Performance was evaluated on 25 validation cases, comparing RAG-LLMs (GPT-5, Claude Sonnet 4.5, DeepSeek-R1, Gemini 2.5 Flash) against their standalone versions and ORAD.
• Diagnostic accuracy was analyzed using Cochran's Q test with post-hoc McNemar's tests and Bonferroni correction.

Main Results:

• RAG-enhanced GPT-5 achieved the highest diagnostic accuracy (20/25), outperforming standalone LLMs (9-13/25) and ORAD (17/25).
• Significant improvement was noted for GPT-5 when integrated with RAG (p=0.002).
• RAG configurations demonstrated superior intra-model stability compared to standalone LLMs, with RAG-GPT-5 achieving 0.90±0.11 stability.

Conclusions:

• JADE, a RAG system, significantly improved diagnostic accuracy and stability for jawbone lesion assessment compared to standalone LLMs.
• This study marks the first application of RAG in dentomaxillofacial radiology, highlighting its value for enhancing AI diagnostic capabilities.
• RAG integration shows promise for advancing AI-assisted diagnostic tools in specialized medical fields.