LViT: Language Meets Vision Transformer in Medical Image Segmentation

Area of Science:

• Medical image analysis
• Deep learning
• Computer vision

Background:

• Medical image segmentation is crucial for diagnosis and treatment planning.
• High-quality labeled data is essential but costly to obtain, limiting model performance.
• Existing models struggle with data scarcity and annotation costs.

Purpose of the Study:

• To propose a text-augmented medical image segmentation model, LViT (Language meets Vision Transformer).
• To address the limitations of insufficient high-quality labeled data in medical image segmentation.
• To enhance segmentation performance using integrated medical text annotations.

Main Methods:

• Developed LViT model incorporating medical text annotations to compensate for image data quality issues.
• Utilized text information for generating improved pseudo-labels in semi-supervised learning.
• Introduced Exponential Pseudo label Iteration (EPI) mechanism and Pixel-Level Attention Module (PLAM) to preserve local image features.
• Designed Language-Vision (LV) loss for supervising unlabeled images using text information.

Main Results:

• LViT demonstrated superior segmentation performance in both fully-supervised and semi-supervised settings.
• Text augmentation effectively compensated for quality deficiencies in medical image data.
• The proposed methods improved pseudo-label generation and feature preservation.

Conclusions:

• LViT offers a promising approach for medical image segmentation by integrating language and vision modalities.
• The model effectively overcomes data annotation cost limitations.
• The developed methodology shows significant potential for advancing medical image analysis.