VT-MFLV: Vision-Text Multimodal Feature Learning V Network for Medical Image Segmentation.

Area of Science:

• Medical Image Analysis
• Artificial Intelligence
• Computer Vision

Background:

• Existing multimodal segmentation methods struggle with integrating medical text for visual feature learning.
• Challenges include insufficient multimodal fusion and low accuracy in fine-grained lesion segmentation.

Purpose of the Study:

• To propose the Vision-Text Multimodal Feature Learning V Network (VT-MFLV) for enhanced medical image segmentation.
• To improve critical lesion recognition accuracy by exploiting the complementarity between medical images and text.

Main Methods:

• The VT-MFLV model incorporates three modules: Diagnostic Image-Text Residual Multi-Head Semantic Encoding (DIT-RMHSE), Fine-Grained Multimodal Fusion Local Attention Encoding (FG-MFLA), and Adaptive Global Feature Compression and Focusing (AGCF).
• These modules facilitate semantic cue preservation, reduce preprocessing complexity, strengthen local cross-modal interaction, and emphasize clinically relevant lesion regions.

Main Results:

• Experiments on pulmonary infection datasets (MosMedData and QaTa-COV1) demonstrated VT-MFLV's effectiveness.
• Achieved Dice scores of 75.61% and 83.34%, and mIoU scores of 63.98% and 72.09% on the respective datasets.
• Performance reached world-leading levels, indicating significant improvements in lesion segmentation.

Conclusions:

• The VT-MFLV model successfully enhances multimodal fusion and lesion segmentation accuracy in medical imaging.
• The proposed architecture offers a promising direction for leveraging text data to improve visual feature learning in medical AI applications.