Depthwise-Dilated Convolutional Adapters for Medical Object Tracking and Segmentation Using the Segment Anything...

Area of Science:

• Medical Imaging
• Deep Learning
• Computer Vision

Background:

• Deep learning drives medical image segmentation but struggles with dynamic scenarios and modality-specific designs.
• Segment Anything Model 2 (SAM2) offers real-time video segmentation but requires extensive data for medical adaptation.
• Existing methods face high computational costs and risk catastrophic forgetting when adapting SAM2 to medical videos.

Purpose of the Study:

• To propose DD-SAM2, an efficient adaptation framework for Segment Anything Model 2 (SAM2) in medical video segmentation and tracking.
• To enhance multi-scale feature extraction for SAM2 using a Depthwise-Dilated Adapter (DD-Adapter) with minimal parameter overhead.
• To enable effective fine-tuning of SAM2 on medical videos using limited training data and leverage its streaming memory for object tracking.

Main Methods:

• Developed DD-SAM2, an efficient adaptation framework for SAM2.
• Incorporated a Depthwise-Dilated Adapter (DD-Adapter) to improve multi-scale feature extraction.
• Utilized SAM2's streaming memory for medical video object tracking and segmentation.

Main Results:

• Achieved superior performance on medical video segmentation and tracking tasks.
• Demonstrated high Dice scores: 0.93±0.04 on TrackRad2025 (tumor segmentation) and 0.97±0.01 on EchoNet-Dynamic (left ventricle tracking).
• Showcased effective fine-tuning of SAM2 on medical videos with limited training data.

Conclusions:

• DD-SAM2 provides an efficient solution for adapting SAM2 to medical video segmentation and tracking.
• The proposed DD-Adapter enhances feature extraction, enabling high performance with minimal parameters.
• This work represents a novel exploration of adapter-based fine-tuning for SAM2 in medical video analysis.