A Language-Guided Progressive Fusion Network with semantic density alignment for Medical Visual Question Answering

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Medical Visual Question Answering (Med-VQA) is crucial for resource-limited settings.
• Existing Med-VQA models struggle with inconsistent information density and dataset biases.
• Addressing these limitations is key to advancing Med-VQA capabilities.

Purpose of the Study:

• To propose a novel Language-Guided Progressive Fusion Network (LGPFN) for Med-VQA.
• To overcome challenges posed by information density disparities and long-tail data distributions.
• To enhance the performance and generalizability of Med-VQA systems.

Main Methods:

• Developed the Language-Guided Progressive Fusion Network (LGPFN).
• Implemented Question-Guided Progressive Multimodal Fusion (QPMF) for feature integration.
• Utilized a Language-Gate Mechanism (LGM) for sample classification (Closed-Ended/Open-Ended).
• Employed Triple Semantic Feature Alignment (TriSFA) for optimizing answer prediction.

Main Results:

• Achieved state-of-the-art performance on multiple Med-VQA datasets.
• Obtained top accuracies: 80.39% (VQA-RAD), 84.07% (SLAKE), 75.74% (PathVQA), 70.60% (VQA-Med 2019).
• Demonstrated the effectiveness and generalizability of the LGPFN framework.

Conclusions:

• The LGPFN model effectively addresses key challenges in Med-VQA.
• The proposed framework shows significant potential for medical AI applications.
• This advancement could contribute to improving universal health coverage through AI-powered medical insights.