On-the-Fly Improving Segment Anything for Medical Image Segmentation Using Auxiliary Online Learning

Area of Science:

• Computer Vision
• Medical Imaging Analysis
• Machine Learning

Background:

• Current Segment Anything Model (SAM) variants, including Medical SAM, exhibit limitations in achieving high accuracy for medical image segmentation.
• Manual or semi-manual corrections by human experts are often required to refine SAM's segmentation predictions on medical images.
• These expert rectifications utilize advanced annotation tools, highlighting a need for automated improvement.

Purpose of the Study:

• To develop a novel approach for enhancing Segment Anything (SA) during test time using online machine learning.
• To improve the segmentation quality of SA on medical images by leveraging expert-provided rectified annotations.
• To create an effective and efficient online learning method suitable for large-scale vision models like SAM.

Main Methods:

• Introduction of Auxiliary Online Learning (AuxOL), a method designed for online learning with large-scale vision models.
• AuxOL incorporates adaptive online-batch processing and adaptive segmentation fusion techniques.
• The approach utilizes rectified annotations from human experts to fine-tune SA models in real-time.

Main Results:

• Experimental validation across eight datasets spanning four medical imaging modalities demonstrated the proposed method's effectiveness.
• The AuxOL approach significantly improved segmentation accuracy for medical images compared to baseline SAM.
• The method proved practical and efficient for enhancing SA on downstream segmentation tasks.

Conclusions:

• Auxiliary Online Learning (AuxOL) offers a practical and effective solution for enhancing the performance of Segment Anything (SA) models in medical image segmentation.
• The proposed method successfully integrates online learning with expert feedback to boost segmentation accuracy.
• This work provides a valuable tool for improving automated segmentation in various medical imaging applications.