LLM-based kidney disease diagnostic framework for Pathologists

Large language models revolutionize the recent paradigm in the medical field and its contributing to various applications, diversified from clinical decision support to information extraction and summarization. The substantial linguistic understanding and contextual awareness allow language models to process and evaluate decision tasks. Concurrently, it addresses the challenges encountered by pathologists in disease diagnosis by adeptly retrieving precise and accurate facts from an external knowledge base. In this paper, we propose a framework which incorporates advanced retrieval augmented generation with prompt engineering techniques, contain prompting levels and structured prompts, which enables the model to extract refine, and customize responses. The model has been equipped with a large corpus of several kidney diseases clinical data which is collected from the vast information sources of kidney diagnostic books. The utilization of varied prompt techniques, exemplified by standard prompts like few-shots and the Reasoning Act (ReAct), manifests notable improvements in disease diagnosis responses. Structured prompts are designed to provide pathologists with specific instructions for formulating questions that effectively enhance the performance of the model. In the evaluation of prompt performance, three key metrics are employed answer relevance, faithfulness, and context relevance. Notably, in the context relevance metric, an optimal performance score of 1.0 was attained indicating perfect alignment with the conversational context.