Gaze-Guided Medical Image Segmentation: A Training-Free Approach using SAM Foundation Model

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Medical image segmentation is crucial but labor-intensive, necessitating automated solutions.
• Deep learning models require extensive labeled data and training, limiting accessibility.
• Foundation models like Segment Anything Model (SAM) offer zero-shot segmentation but often need task-specific adapters.
• Training-free approaches are essential for resource-limited clinical centers.

Purpose of the Study:

• To investigate eye-gaze data as an implicit, efficient prompt for training-free SAM-based medical image segmentation.
• To evaluate gaze-based prompting strategies as a low-cost alternative to manual bounding boxes.
• To demonstrate the clinical relevance and accessibility of gaze-driven segmentation.

Main Methods:

• Utilized eye-gaze data as implicit prompts for the Segment Anything Model (SAM).
• Evaluated multiple gaze-based prompting strategies, including combining bounding boxes with gaze-derived heatmaps.
• Validated the approach on polyp segmentation (Kvasir-SEG) and prostate segmentation (NCI-ISBI 2013).

Main Results:

• Gaze-based prompting achieved satisfactory segmentation results comparable to SAM-based trained models.
• The proposed gaze-based method outperformed segmentation using only manual bounding boxes.
• The most effective strategy involved a combination of bounding boxes and gaze data heatmaps.

Conclusions:

• Eye-gaze data provides a natural, efficient, and low-cost prompting mechanism for foundation models in medical imaging.
• This training-free, gaze-driven approach enhances segmentation automation, reducing annotation time and enabling near real-time applications.
• The method improves accessibility for resource-limited clinical settings, facilitating faster deployment and broader adaptability.