相关实验视频

Updated: Jan 18, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

EndoChat:用于内镜外科手术的接地式多模式大语言模型

Guankun Wang¹, Long Bai², Junyi Wang³

¹The Chinese University of Hong Kong, 999077, Hong Kong Special Administrative Region of China; Theory Lab, Central Research Institute, 2012 Labs, Huawei Technologies Co. Ltd., 999077, Hong Kong Special Administrative Region of China.

Medical image analysis

|September 10, 2025

概括

新的多式大型语言模型 (MLLM) EndoChat 增强了机器人辅助手术培训和决策. 它在内镜手术的理解方面表现出色,提供先进的手术场景分析和对话能力.

科学领域:

人工智能的人工智能
医疗机器人医疗机器人
计算机视觉计算机视觉

背景情况:

多模式大语言模型 (MLLMs) 在计算机辅助诊断和机器人辅助手术方面表现有前途.
在专门的MLLM中存在一个关于内镜外科手术场景理解的差距.

研究的目的:

介绍EndoChat,一个旨在理解内镜手术的MLLM.
开发一个专门的数据集和新的机制,以改善手术场景分析.

主要方法:

使用新型的外科信息提取和注释管道构建了Surg-396K数据集.
实现了一个多尺度的视觉令牌交互机制.
介绍了一种基于视觉对比的推理机制,用于增强学习.

主要成果:

在五个对话范式和七个手术场景理解任务上取得了最先进的表现.
外科医生对EndoChat的对话案例生成提供了积极的反.
证明了在机器人辅助外科手术中推进培训和自动化的潜力.

结论:

在手术应用中,EndoChat代表了MLLM的重大进步.
开发的数据集和模型可以加速机器人辅助手术培训和自动化方面的进展.
数据集和模型的公开可用性有助于进一步的研究和开发.

关键词:

对话范式对话范式内镜外科手术是一种内镜外科手术.多模式的大型语言模型手术场景的理解手术场景的理解

更多相关视频

Mixed Reality Assisted Radical Endoscopic Thyroidectomy

Mixed Reality Assisted Radical Endoscopic Thyroidectomy

Published on: January 31, 2025

Learning Modern Laryngeal Surgery in a Dissection Laboratory

Learning Modern Laryngeal Surgery in a Dissection Laboratory

Published on: March 18, 2020

相关概念视频

Endoscopic Procedures III: Video Capsule Endoscopy

Endoscopic Procedures III: Video Capsule Endoscopy

Capsule endoscopy, or wireless or video capsule endoscopy, is a diagnostic procedure for examining the entire gastrointestinal tract. Patients swallow a capsule about the size of a vitamin tablet. The capsule is equipped with a transmitter, a battery, an LED light source, and a color video camera to capture images throughout the gastrointestinal tract. This procedure is particularly useful for diagnosing conditions such as Crohn's disease, ulcerative colitis, tumors, polyps, ulcers,...

Endoscopic Studies I: Bronchoscopy and Thoracoscopy

Endoscopic Studies I: Bronchoscopy and Thoracoscopy

Endoscopy is a non-surgical medical technique used to examine a person's internal organs and vessels. This lesson will focus on two types of endoscopic studies: bronchoscopy and thoracoscopy.
Bronchoscopy
Description
Bronchoscopy is a procedure that involves direct visualization of the larynx, trachea, and bronchi for diagnostic and therapeutic purposes. A flexible fiber optic or rigid bronchoscope is used to carry out the procedure. The fiber-optic bronchoscope is more frequently used due...