MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research.

Area of Science:

• Biomedical research
• Artificial intelligence in science

Background:

• Scientific research requires complex reasoning over multimodal data, particularly in biology.
• Existing AI benchmarks for multimodal reasoning are limited to college-level difficulty or focus on basic perception, not research-level complexity.
• There's a need for benchmarks that assess advanced multimodal reasoning for scientific discovery.

Purpose of the Study:

• To introduce MicroVQA, a novel visual-question answering (VQA) benchmark.
• To evaluate three key research reasoning capabilities: expert image understanding, hypothesis generation, and experiment proposal.
• To address the gap in research-level multimodal reasoning assessment for AI models.

Main Methods:

• Curated 1,042 multiple-choice questions (MCQs) by biology experts using diverse microscopy data.
• Developed a two-stage MCQ generation pipeline involving an LLM prompt and a 'RefineBot' to eliminate language shortcuts.
• Benchmarked state-of-the-art multimodal large language models (MLLMs) on the MicroVQA dataset.

Main Results:

• State-of-the-art MLLMs achieved a peak performance of 53% on MicroVQA.
• Models with smaller LLMs showed only slight performance degradation, indicating language reasoning is less challenging than multimodal reasoning.
• Fine-tuning with scientific articles improved model performance.
• Analysis revealed perception errors as the most common, followed by knowledge and overgeneralization errors.

Conclusions:

• MicroVQA is a valuable resource for advancing AI-driven biomedical research by assessing critical multimodal reasoning skills.
• Current MLLMs struggle with complex scientific multimodal reasoning, highlighting areas for future development.
• The benchmark provides insights into the types of errors AI models make in scientific contexts.