RAPT: Retrieval-Augmented Visual Prompting with Text-Guidance for Pathological Image Classification.

Area of Science:

• Computational pathology
• Artificial intelligence
• Medical imaging

Background:

• Explainable artificial intelligence (XAI) is crucial for clinical adoption in computational pathology.
• Pre-trained Vision-Language Models (VLMs) show promise but struggle with cancer pathology's complexity.
• Existing visual prompting methods lack clinical specificity or adaptability.

Purpose of the Study:

• To introduce RAPT, a retrieval-augmented and text-guided visual prompting framework.
• To enable explainable pathology classification by integrating visual and textual data.
• To improve the adaptability and clinical relevance of VLM prompting in cancer pathology.

Main Methods:

• Developed RAPT, a framework using retrieval-augmented and text-guided visual prompting.
• Employed semantically related exemplars and class-specific text to create disease-aware prompt tokens.
• Integrated an adaptive weighting mechanism and bridge prompt tokens for robust cue integration.

Main Results:

• RAPT consistently outperformed existing prompting baselines across three cancer pathology datasets.
• Demonstrated clinically actionable insights and robustness to imperfect retrieval.
• Showcased clear failure-mode boundaries, enhancing trustworthiness in decision support.

Conclusions:

• RAPT offers a robust and explainable prompting solution for computational pathology.
• The framework effectively injects diagnostic cues without manual annotation.
• RAPT advances trustworthy AI for pathology by improving interpretability and performance.