D²MNet: Difference-Aware Decoupling and Multi-Prompt Learning for Medical Difference Visual Question Answering

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Computer vision

Background:

• Difference visual question answering (Diff-VQA) is crucial for identifying and reasoning about discrepancies in medical images.
• Current Diff-VQA methods struggle with asymmetric descriptions of image changes and lack task-specific guidance for language models.
• Pretrained language decoders require better integration for effective medical image difference analysis.

Purpose of the Study:

• To introduce D2MNet, a novel framework for medical Diff-VQA that integrates change-aware reasoning with prompt-guided answer generation.
• To address limitations in existing methods by improving the modeling of image differences and enhancing language decoder guidance.
• To enhance the accuracy and contextual relevance of answers generated for medical Diff-VQA tasks.

Main Methods:

• Developed D2MNet, incorporating a Change Analysis Module (CAM) for change detection and prompt generation.
• Implemented a Difference-Aware Module (DAM) utilizing dual attention for fine-grained feature extraction.
• Employed a multi-prompt learning mechanism (MLM) to inject diverse prompts into the language decoder for improved generation.

Main Results:

• D2MNet achieved a CIDEr score of 2.907 ± 0.040 on the MIMIC-DiffVQA benchmark.
• The proposed framework significantly outperformed the strongest baseline, ReAl (2.409).
• Experimental results validate the effectiveness of D2MNet's design for medical Diff-VQA.

Conclusions:

• D2MNet demonstrates superior performance in medical Diff-VQA tasks.
• The framework's approach to change-aware reasoning and prompt-guided generation is effective.
• D2MNet shows promise for advancing difference-aware medical answer generation.