VQAMix: Conditional Triplet Mixup for Medical Visual Question Answering

Area of Science:

• Artificial Intelligence
• Medical Imaging Analysis
• Computer Vision

Background:

• Medical visual question answering (VQA) systems require extensive labeled data, which is costly and time-consuming to acquire.
• Limited labeled data hinders the development and performance of medical VQA models.

Purpose of the Study:

• To introduce VQAMix, a novel data augmentation technique for medical VQA.
• To address the challenges of missing or meaningless answers arising from data augmentation in VQA.

Main Methods:

• VQAMix generates synthetic training samples by linearly combining pairs of existing VQA samples.
• The Learning with Missing Labels (LML) strategy handles missing answers by excluding them.
• The Learning with Conditional-mixed Labels (LCL) strategy uses language priors to ensure meaningful answers for mixed samples.

Main Results:

• VQAMix significantly improved baseline model performance by approximately 7% and 5% on the VQA-RAD and PathVQA benchmarks, respectively.
• The method demonstrated improvements in confidence calibration and model interpretability.
• The proposed VQAMix approach is compatible with various visual-language models.

Conclusions:

• VQAMix is an effective data augmentation strategy for medical VQA, overcoming data limitations.
• The LML and LCL strategies successfully address the challenges of missing and meaningless answers.
• VQAMix offers practical benefits for medical VQA by enhancing performance and interpretability.