DAMON: Difference-Aware Medical Visual Question Answering via Multimodal Large Language Model.

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Multimodal machine learning

Background:

• Difference-aware Medical Visual Question Answering (MVQA) is vital for tracking disease progression and treatment planning.
• Current medical Multimodal Large Language Models (MLLMs) struggle with generalization in difference-aware MVQA.
• Existing datasets are limited to single diseases, failing to capture complex clinical scenarios with multiple diseases.

Purpose of the Study:

• To introduce a large-scale dataset, DAMON-QA, for difference-aware MVQA across multiple diseases.
• To propose a novel Difference-Aware Medical visual questiON answering (DAMON) model.
• To enhance the performance of MLLMs in analyzing visual differences in medical images.

Main Methods:

• Developed DAMON-QA dataset to address limitations of existing single-disease datasets.
• Proposed the DAMON model, trained on DAMON-QA, for difference-aware MVQA.
• Introduced a Disease-driven Prompt Module (DPM) for improved disease identification and difference analysis.

Main Results:

• The DAMON model achieved state-of-the-art (SOTA) performance on the MIMIC-Diff-VQA benchmark.
• The DAMON-QA dataset facilitates better modeling of multi-disease coexistence and overlapping symptoms.
• The DPM effectively guides disease difference analysis in complex medical image inputs.

Conclusions:

• The DAMON model and DAMON-QA dataset represent significant advancements in difference-aware MVQA.
• This work enhances MLLMs' ability to handle complex, multi-disease scenarios in medical image analysis.
• The findings have implications for improving clinical decision-making through more accurate disease progression assessment.