Grounding DINO-US-SAM: Text-Prompted Multiorgan Segmentation in Ultrasound With LoRA-Tuned Vision-Language Models.

Area of Science:

• Medical imaging
• Computer vision
• Artificial intelligence

Background:

• Ultrasound object segmentation faces challenges due to anatomical variability and limited annotated data.
• Existing methods often require organ-specific training, hindering broad applicability.

Purpose of the Study:

• To develop a prompt-driven vision-language model (VLM) for generalizable object segmentation in ultrasound.
• To integrate Grounding DINO with Segment Anything Model 2 (SAM2) for multi-organ ultrasound segmentation.

Main Methods:

• Utilized 18 public ultrasound datasets covering breast, thyroid, liver, prostate, kidney, and paraspinal muscle.
• Fine-tuned Grounding DINO using Low Rank Adaptation (LoRA) on 15 datasets, with 3 held out for testing.
• Evaluated performance against state-of-the-art methods like UniverSeg, MedSAM, and MedCLIP-SAM.

Main Results:

• The proposed VLM approach demonstrated superior performance on most tested ultrasound datasets.
• Achieved strong generalization capabilities on unseen datasets without further fine-tuning.
• Outperformed established segmentation methods including UniverSeg, MedSAM, MedCLIP-SAM, BiomedParse, and SAMUS.

Conclusions:

• Vision-language models show significant promise for scalable and robust ultrasound image analysis.
• The developed method reduces reliance on large, organ-specific annotated datasets.
• Code will be made publicly available at code.sonography.ai upon acceptance.