Med-SCoT: Structured chain-of-thought reasoning and evaluation for enhancing interpretability in medical visual...

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Existing medical visual question answering (Med-VQA) methods prioritize answer accuracy over reasoning.
• This lack of interpretability limits the reliability of Med-VQA in clinical settings.

Purpose of the Study:

• To introduce Med-SCoT, a vision-language model for Med-VQA that incorporates structured chain-of-thought (SCoT) reasoning.
• To improve the interpretability and reliability of Med-VQA systems for clinical applications.

Main Methods:

• Developed Med-SCoT, a model employing a four-stage SCoT reasoning process: Summary, Caption, Reasoning, and Conclusion.
• Proposed a multi-model collaborative correction (CoCo) annotation pipeline for training data.
• Constructed three Med-VQA datasets with structured reasoning chains.
• Introduced SCoTEval, an evaluation framework combining metric-based scores and LLM assessments.

Main Results:

• Med-SCoT achieved advanced answer accuracy in Med-VQA tasks.
• The model generated structured, clinically aligned, and logically coherent reasoning chains.
• SCoTEval demonstrated high agreement with expert judgments, confirming its reliability for assessing reasoning quality.

Conclusions:

• Med-SCoT effectively addresses the need for interpretable reasoning in Med-VQA.
• The SCoT approach and SCoTEval framework offer a promising direction for developing more reliable clinical AI tools.