WoundcareVQA: A multilingual visual question answering benchmark dataset for wound care

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Natural Language Processing

Background:

• Consumer health questions regarding wound care are prevalent online.
• Existing AI models lack specialized capabilities for multimodal, multilingual wound care VQA.

Purpose of the Study:

• To introduce and establish the task of wound care multimodal multilingual visual question answering (VQA).
• To benchmark the performance of current AI models on this novel task.
• To identify key areas for future research and development in wound care VQA.

Main Methods:

• A dataset of wound care VQA was created using online consumer health questions.
• Medical doctors provided expert metadata and response labels for the dataset.
• Instruct-enabled multilingual VQA models (GPT-4o, Gemini-1.5-Pro, Qwen-VL) were evaluated.
• Automatic evaluation metrics were compared against domain expert ratings.

Main Results:

• A dataset comprising 477 wound care cases, 768 responses, 748 images, and extensive metadata was constructed.
• Metadata classification accuracy ranged from 0.32-0.78.
• Response generation performance metrics (BLEU, BERTScore, ROUGE-L) were reported for English and Chinese.

Conclusions:

• The study successfully constructed and explored the task of multimodal, multilingual VQA for wound care.
• The findings aim to stimulate further research in wound care metadata classification, VQA response generation, and evaluation methodologies.