Medical Visual Question Answering via Conditional Reasoning and Contrastive Learning.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Medical visual question answering (Med-VQA) is crucial for healthcare but faces challenges.
• Diverse clinical questions and complex medical images hinder current Med-VQA development.
• Small-scale datasets limit the training of effective visual feature extractors.

Purpose of the Study:

• To address the limitations in Med-VQA development.
• To propose a novel conditional reasoning mechanism for adaptive reasoning skills.
• To introduce pre-training a visual feature extractor using contrastive learning on large unlabeled datasets.

Main Methods:

• Developed a conditional reasoning mechanism with question-conditioned and type-conditioned components.
• Implemented a strategy for adaptive learning of reasoning skills tailored to Med-VQA tasks.
• Pre-trained a visual feature extractor via contrastive learning on extensive unlabeled radiology images.

Main Results:

• Achieved significant improvements in prediction accuracy on Med-VQA benchmarks.
• Demonstrated superior performance compared to existing state-of-the-art methods.
• Validated the effectiveness of the proposed conditional reasoning and pre-training strategies.

Conclusions:

• The proposed approach enhances Med-VQA performance by improving reasoning capabilities.
• Pre-training visual feature extractors on large unlabeled datasets is effective for Med-VQA.
• This work advances the development of accurate and reliable Med-VQA systems.