Prompt-guided foundation model tuning for pathology image classification

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Histopathology image analysis

Background:

• Foundation models are crucial for whole slide image (WSI) classification in computational pathology.
• Existing methods often use frozen pre-trained models, ignoring domain shift and task discrepancies.
• This limits the effectiveness of general models on specialized histopathology data.

Purpose of the Study:

• To propose a novel framework, Prompt-guided Adaptive Model Transformation (PAMT), for adapting foundation models to histopathology.
• To address the domain shift and task discrepancy challenges in computational pathology.
• To enhance the performance of foundation models for WSI classification in this domain.

Main Methods:

• Introduced Representative Patch Sampling (RPS) and Prototypical Visual Prompt (PVP) to create informative representations of histopathological data.
• Implemented Adaptive Model Transformation (AMT) using adapter modules to bridge the domain gap.
• Enabled foundation models to acquire domain-specific features through targeted adaptation.

Main Results:

• PAMT demonstrated consistent and substantial improvements in classification accuracy across 14 public datasets.
• The framework effectively adapted general foundation models to the specific domain of histopathology.
• Achieved new benchmark performance for pathology image classification tasks.

Conclusions:

• PAMT offers a powerful new approach for computational pathology, significantly enhancing WSI classification.
• Targeted model adaptation is critical for leveraging foundation models in specialized domains like histopathology.
• The proposed methods provide a robust solution for domain shift challenges in medical image analysis.