大型语言模型驱动的内镜分析和报告生成视频-试点研究
Davide Massimi1, Luca Di Stefano1, Tommy Rizkala1
1IRCCS Humanitas Research Hospital, Milan, Italy.
概括
多模式大语言模型 (MLLMs) 在生成临床食道胃管透镜 (EGD) 报告方面表现不够好. 需要进一步的研究来优化MLLMs,以便准确的EGD分析和报告.
科学领域:
- 人工智能在医学中的应用
- 医学成像分析 医学成像分析
- 胃肠病学 胃肠病学
背景情况:
- 多模式大语言模型 (MLLMs) 为自动化临床视频分析提供了潜力.
- 在全食管胃管腺镜 (EGD) 分析中MLLM的有效性和计算机辅助检测/诊断 (CAD) 叠加的影响尚未得到充分证实.
研究的目的:
- 评估MLLM (Gemini 2.5 Pro) 在临床上生成足够的EGD报告的能力.
- 为了确定屏幕上的CAD重叠对EGD分析中MLLM性能的影响.
主要方法:
- 五个完整的EGD视频被Gemini 2.5 Pro在干净和CAD覆盖版本中分析.
- 五位盲目内镜医生评估了报告的完整性,可视化和病变特征的充分性.
- 两名专家内镜师使用时间窗口规则评估了MLLM对地标和病变的准确性.
主要成果:
- 在56.0%的清洁视频中,MLLM的完整性被评为足够的,而在CAD重叠的视频中,这一比例为48.0% (p=0.500).
- 在清洁和叠加视频 (p=1.000) 之间,可视化和损伤特征评级是相同的.
- 与清洁视频 (0.55) (p=0.029) 相比,CAD重叠 (0.33) 的总体里程碑协议准确性明显较低.
结论:
- 在这项试点研究中,Gemini 2.5 Pro在临床EGD报告中表现不够好.
- 在CAD重叠的存在对MLLM的地标检测准确性产生了负面影响.
- 在MLLM部署用于EGD分析之前,需要进行大量的优化和大规模验证.
相关概念视频
Endoscopic Procedures III: Video Capsule Endoscopy
1.1K
Capsule endoscopy, or wireless or video capsule endoscopy, is a diagnostic procedure for examining the entire gastrointestinal tract. Patients swallow a capsule about the size of a vitamin tablet. The capsule is equipped with a transmitter, a battery, an LED light source, and a color video camera to capture images throughout the gastrointestinal tract. This procedure is particularly useful for diagnosing conditions such as Crohn's disease, ulcerative colitis, tumors, polyps, ulcers,...
1.1K
Imaging Studies III: Gastrointestinal Motility Studies and Virtual Colonoscopy
555
This lesson explores three gastrointestinal imaging techniques: radionuclide testing, colonic transit studies, and virtual colonoscopy.
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
555
Endoscopic Procedures II: Colonoscopy
999
The colon, or large intestine, is the final segment of the digestive system. Its primary functions include absorbing water and vitamins produced by gut bacteria and transforming waste from liquid to solid to form stool. In adults, the large intestine is approximately 5 feet long and consists of four main sections:
999


