LLM-based kidney disease diagnostic framework for Pathologists

Area of Science:

• Artificial Intelligence in Medicine
• Medical Informatics
• Natural Language Processing

Background:

• Large language models (LLMs) are transforming medical applications, including clinical decision support and information extraction.
• LLMs offer advanced linguistic understanding and contextual awareness for complex medical tasks.
• Pathologists face challenges in disease diagnosis, necessitating efficient retrieval of accurate information.

Purpose of the Study:

• To propose a framework using retrieval augmented generation and prompt engineering for medical diagnosis.
• To enhance the extraction, refinement, and customization of responses for pathologists.
• To improve the accuracy and relevance of disease diagnosis using LLMs.

Main Methods:

• Developed a framework integrating retrieval augmented generation with prompt engineering techniques.
• Utilized prompting levels and structured prompts, including few-shot and Reasoning Act (ReAct) methods.
• Trained the model on a large corpus of kidney disease clinical data from diagnostic books.
• Employed answer relevance, faithfulness, and context relevance metrics for evaluation.

Main Results:

• The framework demonstrated notable improvements in disease diagnosis responses.
• Structured prompts were designed to guide pathologists in formulating effective questions.
• Achieved an optimal performance score of 1.0 in context relevance, indicating perfect alignment.

Conclusions:

• The proposed framework effectively leverages LLMs and prompt engineering for enhanced medical diagnosis.
• Advanced prompt techniques significantly improve the performance of LLMs in retrieving and presenting disease information.
• The system shows promise in supporting pathologists by providing accurate and contextually relevant diagnostic information.