TSMS-SAM2:多尺度时间采样增强和内存分割修剪,用于手术场景中的快速视频对象分割和跟踪.
Guoping Xu1, Hua-Chieh Shao1, You Zhang1
1The Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
概括
本研究介绍了TSMS-SAM2,这是一个新的外科视频对象细分和跟踪框架. 它通过解决基础模型中的快速运动和内存问题来提高准确性和效率.
科学领域:
- 计算机视觉 计算机视觉
- 医疗成像医学成像
- 手术机器人手术机器人手术机器人手术机器人
背景情况:
- 像SegmentAnything Model 2 (SAM2) 这样的基础模型具有先进的提示式视频对象分割和跟踪 (VOST).
- 由于SAM2中复杂的运动和内存冗余,将VOST应用于外科视频是很困难的,这阻碍了有效的学习.
- 现有的方法与手术环境的动态和复杂性质作斗争.
研究的目的:
- 通过开发一个新的框架,TSMS-SAM2.2,来增强手术视频中的快速VOST.
- 为应对SAM2中快速物体运动和内存冗余所带来的挑战,用于外科应用.
- 在复杂的外科手术场景中提高细分的稳定性和效率.
主要方法:
- 拟议的TSMS-SAM2框架具有多时间尺度的视频采样增强运动变异性.
- 实现了内存分割和修剪机制,以组织和过过去的功能.
- 使用EndoVis2017和EndoVis2018数据集进行评估.
主要成果:
- TSMS-SAM2实现了最先进的性能,平均子得分在EndoVis2017上为95.24±0.96%,在EndoVis2018上为86.73±15.46%,在EndoVis2018上为86.73±15.46%.
- 在手术视频细分方面表现优于以前基于SAM和特定任务的方法.
- 废除研究验证了多尺度时间增强和记忆分割的有效性.
结论:
- 在复杂的手术视频中,TSMS-SAM2提供了一个强大而高效的解决方案,用于提示性VOST.
- 拟议的框架通过解决运动动态和内存冗余,显著提高了细分精度.
- 这些发现突出了TSMS-SAM2在推进计算机辅助手术和分析方面的潜力.
相关概念视频
pH Scale
79.4K
Hydronium and hydroxide ions are present both in pure water and in all aqueous solutions, and their concentrations are inversely proportional as determined by the ion product of water (Kw). The concentrations of these ions in a solution are often critical determinants of the solution’s properties and the chemical behaviors of its other solutes. Two different solutions can differ in their hydronium or hydroxide ion concentrations by a million, billion, or even trillion times. A common means of...
79.4K
System of Memory
7.3K
Memory is categorized into three major systems: sensory memory, short-term memory (STM), and long-term memory (LTM). These systems differ in their capacity and the duration for which they can hold information. Sensory memory captures raw sensory input from the environment, holding it for just a few seconds or less. For example, on hearing a brief, loud sound, like a car horn honking, the sound seems to linger in the mind for a moment even after it stops. This is an instance of sensory memory...
7.3K
Working Memory
852
Working memory refers to a combination of components, including short-term memory and attention, that allow an individual to hold information temporarily as we perform cognitive tasks. It is an essential cognitive function that enables the execution of complex tasks such as problem-solving, comprehension, and reasoning. Unlike short-term memory, which simply involves the storage of information for a brief period, working memory involves the active manipulation and processing of this...
852
¹H NMR: Complex Splitting
1.9K
A proton M that is coupled to a proton X results in doublet signals for M. However, NMR-active nuclei can be simultaneously coupled to more than one nonequivalent nucleus. When M is coupled to a second proton A, such as in styrene oxide, each peak in the doublet is split into another doublet.
Splitting diagrams or splitting tree diagrams are routinely used to depict such complex couplings. While drawing splitting diagrams, the splitting with the larger coupling constant is usually applied...
Splitting diagrams or splitting tree diagrams are routinely used to depict such complex couplings. While drawing splitting diagrams, the splitting with the larger coupling constant is usually applied...
1.9K
Potential Due to a Polarized Object
770
A neutral atom consists of a positively charged nucleus surrounded by a negatively charged electron cloud. When placed in an external electric field, the external electric force pulls the electrons and nucleus apart, opposite to the intrinsic attraction between the nucleus and the electrons. The opposing forces balance each other with a slight shift between the center of masses of the nucleus and the electron cloud, resulting in a polarized atom. On the other hand, a few molecules, like water,...
770
Potential Due to a Magnetized Object
792
Magnetic dipoles in magnetic materials are aligned when placed under an external magnetic field. For paramagnets and ferromagnets, dipole alignment occurs in the direction of the magnetic field. However, the dipoles align opposite to the field in the case of diamagnets. This state of magnetic polarization due to the external field is called magnetization. Magnetization is defined as the dipole moment per unit volume. It plays a similar role to polarization in electrostatics.
The vector...
The vector...
792


