Anomaly Matters: An Anomaly-Oriented Model for Medical Visual Question Answering

Area of Science:

• Medical Imaging Analysis
• Computer Vision
• Artificial Intelligence

Background:

• Medical images contain abnormalities crucial for diagnosing diseases.
• Current generic Visual Question Answering (VQA) frameworks are insufficient for specific medical needs.
• Effective VQA requires understanding and localizing anomalies within medical images.

Purpose of the Study:

• To develop novel, medical-specific modules for Visual Question Answering (VQA) that leverage anomaly localization.
• To enhance the performance of VQA systems in medical domains, addressing limitations of generic approaches.
• To improve the accuracy and applicability of AI in interpreting medical images for clinical practice.

Main Methods:

• Introduction of two novel modules: multiplication anomaly sensitive module and residual anomaly sensitive module.
• Utilizing weakly supervised anomaly localization information to guide VQA.
• Integration of a transformer decoder and multi-task learning strategy for enhanced reasoning and generalization.

Main Results:

• The multiplication anomaly sensitive module masks image features based on anomaly maps for anomaly-related questions.
• The residual anomaly sensitive module learns flexible anomaly features while retaining original image information for anomaly-unrelated questions.
• Experiments on diverse medical datasets demonstrated the superiority of the proposed methods over existing state-of-the-art approaches.

Conclusions:

• The proposed medical-specific modules significantly advance medical Visual Question Answering capabilities.
• The novel approach improves the ability to answer both anomaly-related and anomaly-unrelated questions accurately.
• This work offers a more practical and effective AI solution for clinical medical image analysis.