Bilateral network with text guided aggregation architecture for lung infection image segmentation.

Area of Science:

• Medical image analysis
• Artificial intelligence in healthcare
• Computer vision

Background:

• Lung image segmentation is critical for diagnosing illnesses but current methods struggle with varied infection shapes and sizes.
• Existing multimodal approaches combining text and image data are often inefficient and ineffective due to network limitations.

Purpose of the Study:

• To develop an accurate and efficient lung image segmentation method by integrating diagnostic reports and visual information.
• To overcome the limitations of existing multimodal networks in terms of speed and performance.

Main Methods:

• Proposed a Bilateral Network with Text Guided Aggregation Architecture (BNTGAA) for comprehensive fusion of local and global information.
• Implemented a global fusion branch using Hadamard product for text-vision feature alignment.
• Utilized a multi-scale cross-fusion branch with positional coding and skip connections, feeding into a Mamba module for efficient segmentation.

Main Results:

• The BNTGAA demonstrated superior accuracy and efficiency in quantitative and qualitative evaluations.
• Achieved significant improvements in mIoU (3.08%) and Dice scores (2.35%) on the QaTa-COVID19 dataset.
• Outperformed existing multimodal networks in computational speed and showed strong performance even with partial training data.

Conclusions:

• The proposed fusion architecture, powered by the Mamba backbone, effectively enhances lung image segmentation accuracy and efficiency.
• Text-guided aggregation across multiple scales is key to the architecture's improved performance.
• The developed BNTGAA offers a promising solution for automated lung illness understanding.