Pathology-CoT: learning visual chain-of-thought agents from expert whole-slide image diagnosis behaviour

Area of Science:

• Digital Pathology
• Artificial Intelligence in Medicine
• Computational Pathology

Background:

• Whole-slide image diagnosis is complex, requiring expert navigation and reasoning.
• Current AI systems lack the experiential viewing behaviors of pathologists, limiting their diagnostic capabilities.
• Bridging the gap between expert tacit knowledge and AI training data is crucial for advancing diagnostic AI.

Purpose of the Study:

• To introduce Pathology-CoT, a novel framework for creating scalable agent supervision from expert pathologist viewing behavior.
• To develop Pathology-o3, a two-stage agent capable of proposing regions of interest and performing behavior-guided reasoning for diagnostic tasks.
• To improve the accuracy and explainability of AI systems in whole-slide image analysis.

Main Methods:

• An AI session recorder captured and standardized expert pathologist navigation logs into behavioral commands and bounding boxes.
• A human-in-the-loop review pipeline refined AI-generated rationales into 'where to look' and 'why it matters' supervision, accelerating labeling sixfold.
• A two-stage agent, Pathology-o3, was trained using the generated supervision data for region proposal and behavior-guided reasoning.

Main Results:

• Pathology-o3 demonstrated superior performance in gastrointestinal lymph node metastasis detection compared to state-of-the-art vision-language models.
• The framework showed consistent performance improvements across various vision-language model backbones.
• Strong performance was maintained on an independent external validation cohort, indicating robustness and generalizability.

Conclusions:

• Pathology-CoT effectively translates expert pathologist behavior into valuable AI training data.
• The developed Pathology-o3 agent significantly enhances diagnostic accuracy and explainability in digital pathology.
• This approach represents a significant advancement in developing practical agentic systems for complex medical image diagnosis.