Language-Guided Segmentation of Medical Images: A Review of Foundation Models

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Vision-language foundation models (VLMs) integrate image encoders and text prompts for versatile medical image segmentation.
• These models enable segmentation of various anatomical structures, lesions, and modalities via natural language.
• Recent advancements include contrastive pretraining and models like Segment Anything Model (SAM) and its medical adaptations.

Purpose of the Study:

• To provide a comprehensive survey of VLMs for medical image segmentation.
• To categorize existing VLMs and discuss their adaptation strategies.
• To identify current challenges and propose a future research roadmap.

Main Methods:

• Literature review and synthesis of VLMs in medical image segmentation.
• Categorization of models into text-prompt-guided, LLM-embedded, and hybrid frameworks.
• Analysis of adaptation techniques (fine-tuning, LoRA, adapters, prompt engineering) across different imaging modalities.

Main Results:

• Organized literature by modality (CT, MRI, pathology, radiography, ultrasound) and clinical use (organ segmentation, tumor delineation, radiotherapy planning).
• Summarized evaluation metrics and benchmark datasets.
• Identified key challenges: prompt dependence, mask hallucination, slow inference, and data limitations.

Conclusions:

• VLMs offer significant potential for medical image segmentation, transforming clinical workflows.
• Addressing challenges like prompt dependence and inference speed is crucial for clinical translation.
• Future research should focus on trustworthy deployment, multimodal pretraining, and seamless clinical integration.