Parallel multi-head attention and term-weighted question embedding for medical visual question answering.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Medical Visual Question Answering (Med-VQA) models face challenges due to the unique nature of medical images and the scarcity of large-scale labeled datasets.
• Existing Med-VQA approaches often depend on transfer learning and cross-modal fusion, requiring external data for effective feature representation.

Purpose of the Study:

• To develop a novel parallel multi-head attention framework (MaMVQA) for Med-VQA that does not require external training data.
• To address the limitations of general domain Visual Question Answering (VQA) models in the medical context.

Main Methods:

• Image feature extraction is performed using an unsupervised Denoising Auto-Encoder (DAE).
• Language feature extraction utilizes term-weighted question embedding, enhanced by a supervised term-weighting (STW) scheme called qf-MI, based on mutual information (MI).

Main Results:

• The MaMVQA framework achieved state-of-the-art performance on the VQA-RAD benchmark without external data.
• The model demonstrated significantly improved accuracy for both close-ended (78.68%) and open-ended (55.31%) questions.

Conclusions:

• The proposed MaMVQA framework offers an effective solution for Med-VQA, overcoming data limitations.
• The study highlights the significance of individual components through extensive ablation studies, validating the framework's design.