利用大规模的语言和视觉模型从大规模的图像文本结肠镜记录中提取知识
Shuo Wang1,2,3,4, Yan Zhu5,6, Zhiwei Yang7,8,9
1Digital Medical Research Centre, School of Basic Medical Sciences, Fudan University, Shanghai, China. shuowang@fudan.edu.cn.
Nature biomedical engineering
|September 16, 2025
概括
这项研究介绍了EndoKED,一种人工智能方法,可以自动从结肠镜记录中提取详细的息肉信息. 这种方法显著改善了聚的检测和细分,以获得更好的胃肠病学人工智能.
科学领域:
- 医疗成像医学成像
- 人工智能的人工智能
- 胃肠病学 胃肠病学
背景情况:
- 结肠镜分析的AI需要大型,多样化的,专家注释的数据集,这些数据集往往是有限的.
- 常规的临床结肠镜记录提供了大量的图像文本数据,但注释需要大量的劳动力.
- 现有的方法与数据集限制作斗争,阻碍了AI模型的性能和通用化.
研究的目的:
- 开发一个自动化数据挖掘范式,EndoKED,从结肠镜记录中提取深度知识.
- 为人工智能培训,将原始结肠镜图像文本数据转化为像素级注释数据集.
- 使用人工智能增强聚检测,细分和光学活检能力.
主要方法:
- 利用大型语言和视觉模型的进步.
- 提出 EndoKED,一个用于知识提取和蒸的数据挖掘范式.
- 从原始结肠镜记录自动创建像素级注释图像数据集.
主要成果:
- 在报告和图像层面上,EndoKED在检测息肉方面表现出卓越的性能.
- 实现了像素级的息肉注释,显著改善了细分模型.
- 使用EndoKED的预训练导致了在聚细分方面最先进的性能和概括.
- EndoKED视觉骨干能够在光学活检中实现专家级别的性能,并通过数据有效的学习.
结论:
- EndoKED有效地自动化了从临床结肠镜记录中创建注释数据集.
- 拟议的方法显著提升了人工智能在聚检测,细分和光学活检方面的能力.
- EndoKED有助于开发更强大的和可通用的人工智能工具,用于结肠镜分析.
相关概念视频
Imaging Studies III: Gastrointestinal Motility Studies and Virtual Colonoscopy
379
This lesson explores three gastrointestinal imaging techniques: radionuclide testing, colonic transit studies, and virtual colonoscopy.
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
Radionuclide Testing
Radionuclide testing is a sophisticated medical technique for assessing gastrointestinal motility. It focuses on gastric emptying and colonic transit time. Radioactive markers track the movement of food through the digestive system, providing insights into gastrointestinal disorders.
In gastric emptying studies, a meal's liquid and...
379
Endoscopic Procedures II: Colonoscopy
632
The colon, or large intestine, is the final segment of the digestive system. Its primary functions include absorbing water and vitamins produced by gut bacteria and transforming waste from liquid to solid to form stool. In adults, the large intestine is approximately 5 feet long and consists of four main sections:
632
Endoscopic Procedures III: Video Capsule Endoscopy
731
Capsule endoscopy, or wireless or video capsule endoscopy, is a diagnostic procedure for examining the entire gastrointestinal tract. Patients swallow a capsule about the size of a vitamin tablet. The capsule is equipped with a transmitter, a battery, an LED light source, and a color video camera to capture images throughout the gastrointestinal tract. This procedure is particularly useful for diagnosing conditions such as Crohn's disease, ulcerative colitis, tumors, polyps, ulcers,...
731
Endoscopic Procedures IV: Sigmoidoscopy and Laproscopy
574
Sigmoidoscopy and laparoscopy are distinct medical procedures that enable physicians to internally inspect different parts of the GI tract. Although they serve different purposes, each is essential for diagnosing and, in some cases, treating various medical conditions.
Sigmoidoscopy
Sigmoidoscopy is a diagnostic procedure that uses a flexible sigmoidoscope equipped with a light source and camera to examine the rectum and sigmoid colon. The procedure involves inserting the tube through the anus...
Sigmoidoscopy
Sigmoidoscopy is a diagnostic procedure that uses a flexible sigmoidoscope equipped with a light source and camera to examine the rectum and sigmoid colon. The procedure involves inserting the tube through the anus...
574


