Medical question answering: A comprehensive multimodal and LLM-driven survey

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing
• Computer Vision

Background:

• Medical Question Answering (MQA) is crucial for evidence-based medical information retrieval.
• Advancements in AI, NLP, computer vision, and LLMs are transforming MQA from text-only to multimodal systems.

Purpose of the Study:

• To provide a comprehensive review of text-based and image-based MQA systems.
• To analyze applications, datasets, and modeling paradigms in MQA.
• To identify challenges and future research directions for reliable MQA.

Main Methods:

• Systematic literature review of MQA systems.
• Development of a unified taxonomy for MQA tasks (scientific, clinical, consumer, examination).
• Analysis of text-based and vision-based MQA datasets and methodologies, including LLM-driven approaches.

Main Results:

• MQA systems are rapidly evolving towards multimodal and LLM-based frameworks, especially for medical visual question answering.
• Existing datasets and models show progress but have limitations in generalization, reasoning, and clinical applicability.
• Key challenges include reliability, hallucination, explainability, fairness, and clinical safety.

Conclusions:

• The evolution of MQA systems towards multimodal and LLM-based approaches is significant.
• Further research is needed to address limitations in data quality, reasoning, evaluation, and real-world deployment.
• Developing trustworthy and clinically applicable MQA systems requires focused efforts on identified open research directions.