CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection.

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Computational anatomy

Background:

• Public datasets have advanced automated organ segmentation and tumor detection.
• Current models struggle with limited data, partial labeling, and diverse tumor types, restricting their generalization.
• Existing methods often overlook anatomical semantics, hindering extension to new domains.

Purpose of the Study:

• To develop a universal segmentation model that overcomes limitations of dataset-specific approaches.
• To incorporate text embeddings from Contrastive Language-Image Pre-training (CLIP) for improved anatomical understanding.
• To achieve robust segmentation of multiple organs and tumor types across varied medical imaging data.

Main Methods:

• Proposed a CLIP-Driven Universal Model integrating CLIP's text embeddings into segmentation architectures.
• Developed a structured feature embedding by encoding anatomical relationships via CLIP.
• Trained the model on 14 diverse datasets (3,410 CT scans) and evaluated on 3 external datasets (6,162 CT scans).

Main Results:

• Achieved state-of-the-art performance, ranking first on the Medical Segmentation Decathlon (MSD) leaderboard.
• Successfully segmented 25 organs and 6 types of tumors with high accuracy.
• Demonstrated superior computational efficiency (6× faster) and better generalization to varied CT scan sites.

Conclusions:

• The CLIP-Driven Universal Model offers a powerful, efficient, and generalizable solution for medical image segmentation.
• CLIP-based label encoding effectively captures anatomical semantics, enabling broader application.
• The model shows strong transfer learning capabilities for novel segmentation tasks.