Related Experiment Video
Updated: Sep 9, 2025
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
DeepGut: A collaborative multimodal large language model framework for digestive disease assisted diagnosis and
Xiao-Han Wan1, Mei-Xia Liu1, Yan Zhang1
1Department of Gastroenterology, Qilu Hospital of Shandong University, Jinan 250012, Shandong Province, China.
Background:
Gastrointestinal diseases have complex etiologies and clinical presentations. An accurate diagnosis requires physicians to integrate diverse information, including medical history, laboratory test results, and imaging findings. Existing artificial intelligence-assisted diagnostic tools are limited to single-modality information, resulting in recommendations that are often incomplete and may be associated with clinical or legal risks.
Aim:
To develop and evaluate a collaborative multimodal large language model (LLM) framework for clinical decision-making in digestive diseases.
Methods:
In this observational study, DeepGut, a multimodal LLM collaborative diagnostic framework, was developed to integrate four distinct large models into a four-tiered structure. The framework sequentially accomplishes multimodal information extraction, logical "chain" construction, diagnostic and treatment suggestion generation, and risk analysis. The model was evaluated using objective metrics, which assess the reliability and comprehensiveness of model-generated results, and subjective expert opinions, which examine the effectiveness of the framework in assisting physicians.
Results:
The diagnostic and treatment recommendations generated by the DeepGut framework achieved exceptional performance, with a diagnostic accuracy of 97.8%, diagnostic completeness of 93.9%, treatment plan accuracy of 95.2%, and treatment plan completeness of 98.0%, significantly surpassing the capabilities of single-modal LLM-based diagnostic tools. Experts evaluating the framework commended the completeness, relevance, and logical coherence of its outputs. However, the collaborative multimodal LLM approach resulted in increased input and output token counts, leading to higher computational costs and extended diagnostic times.
Conclusion:
The framework achieves successful integration of multimodal diagnostic data, demonstrating enhanced performance enabled by multimodal LLM collaboration, which opens new horizons for the clinical application of artificial intelligence-assisted technology.
Related Concept Videos
01:30Inflammatory Bowel Disease III: Diagnostic Studies and Management I-Nutritional Therapy
Diagnostic studies
A colonoscopy is the definitive screening test, distinguishing ulcerative colitis from other colon diseases with similar symptoms. During a colonoscopy test, inflamed mucosa with exudate ulcerations can be observed, and biopsies are taken to determine the histologic characteristics of the...
01:26Imaging Studies III: Gastrointestinal Motility Studies and Virtual Colonoscopy
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
01:29Chronic Pancreatitis II: Collaborative Care
Assessment:
01:20Gastrointestinal Motility Disorders
01:17Chronic Bowel Disorders: Introduction
Irritable Bowel Syndrome (IBS) is a common disorder affecting the gastrointestinal tract. The distinctive feature is recurrent abdominal pain associated with altered bowel movements, manifesting as constipation, diarrhea, or fluctuating between both. The...
01:28Peptic Ulcer Disease III: Clinical Manifestations and Diagnostic Studies
Few clinical manifestations differentiate gastric ulcers from duodenal ulcers. Distinctions in the location, timing, and pain relief are crucial for healthcare providers in differentiating between gastric and duodenal ulcers during clinical assessments.