Advancing Accuracy in Multimodal Medical Tasks Through Bootstrapped Language-Image Pretraining (BioMedBLIP):...

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Medical image analysis is vital for diagnosis and education.
• Visual Question Answering (VQA) and image captioning are key applications.
• Current models require domain-specific adaptations for optimal performance.

Purpose of the Study:

• Introduce BioMedBLIP models, fine-tuned for medical VQA tasks.
• Evaluate BioMedBLIP against the original Bootstrapping Language-Image Pretraining (BLIP) model.
• Utilize specialized medical datasets like ROI and MIMIC-CXR for fine-tuning.

Main Methods:

• Developed 9 BioMedBLIP versions across 3 downstream tasks and multiple datasets.
• Pretrained the BLIP model using diverse medical datasets.
• Trained models for varying epochs to optimize performance.

Main Results:

• BioMedBLIP models surpassed state-of-the-art (SOTA) in VQA generation on SLAKE, VQA-RAD, and ICF datasets.
• Achieved superior performance in VQA classification on SLAKE and competitive results on VQA-RAD and PathVQA.
• Outperformed SOTA in image captioning tasks, demonstrating the value of medical pretraining.
• Excelled in 75% of 20 task/dataset combinations, setting new SOTA in 15.

Conclusions:

• BioMedBLIP models show significant potential for advancing medical image analysis.
• Pretraining with domain-specific medical data enhances model performance.
• The models contribute to AI in medicine, aiding diagnosis, education, and research.