Segment Any Tissue: One-shot reference guided training-free automatic point prompting for medical image segmentation

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Medical image segmentation faces high annotation costs and challenges in adapting to new tasks.
• Existing visual foundation models struggle with automatic, high-quality prompt generation for class-agnostic medical image segmentation.

Purpose of the Study:

• To introduce Segment Any Tissue (SAT), a training-free framework for automatic, class-agnostic medical image segmentation.
• To address the challenges of high annotation costs and task adaptation in medical image segmentation.

Main Methods:

• SAT employs a dual-space cyclic prompt engineering approach, integrating feature and physical space distance metrics for automatic prompt generation.
• It leverages pretrained foundation models for feature matching and a class-agnostic segmentation model for results.
• An ensemble version of SAT incorporates multiple reference images to improve performance.

Main Results:

• SAT was validated on six diverse medical segmentation tasks, demonstrating effectiveness across macroscopic and microscopic scales.
• Ablation studies confirmed SAT's ability to handle various tissue sizes and the efficacy of its components.
• Comparative experiments showed SAT matches or surpasses fully supervised methods and outperforms existing one-shot techniques.

Conclusions:

• SAT offers a training-free, one-shot solution for medical tissue segmentation, drastically reducing annotation requirements.
• The framework enhances task transferability and lays the groundwork for intelligent medicine in clinical settings.