Bridging multi-level gaps: Bidirectional reciprocal cycle framework for text-guided label-efficient segmentation in...

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Computer vision

Background:

• Text-guided visual understanding offers a promising approach for echocardiography, reducing the need for extensive labeled datasets by embedding clinical information into visual tasks.
• Contrastive language-image pretraining (CLIP) methods adapt pre-trained networks using image-text features but face challenges with multi-level gaps (spatial, contextual, domain) in medical imaging.

Purpose of the Study:

• To address the multi-level image-text gaps in echocardiography for improved downstream task learning.
• To develop a novel framework that effectively bridges these gaps for dense prediction tasks.

Main Methods:

• Proposed a bidirectional reciprocal cycle (BRC) framework to align global and local image-text features through pyramid reciprocal alignments.
• Implemented a consistency enforcement for forward and reverse mappings (text feature, feature text/image) to capture contextual relationships.
• Integrated a cross-modal attention mechanism for adapting to specific segmentation tasks and guiding them with complex text information.

Main Results:

• Achieved state-of-the-art performance on segmentation tasks with a Dice Similarity Coefficient (DSC) of 95.2%, outperforming 22 existing methods.
• Demonstrated significant accuracy improvements and reduced reliance on labeled data, with DSC increasing from 81.5% to 86.6% using only 1% labeled data with text assistance.
• Validated extensive experiments on 11,048 patients.

Conclusions:

• The BRC framework effectively bridges multi-level image-text gaps in echocardiography.
• BRC significantly enhances segmentation accuracy and data efficiency in medical image analysis.
• This approach holds potential for advancing clinical task learning with reduced labeling requirements.