When multiple instance learning meets foundation models: Advancing histological whole slide image analysis

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Digital pathology

Background:

• Deep multiple instance learning (MIL) is standard for whole slide image (WSI) classification.
• The comparative performance of different foundation models (FMs) and MIL methods for WSI analysis is not well-established.
• Variations in patch-level embeddings and slide-level aggregation strategies complicate comparisons.

Purpose of the Study:

• To systematically compare the performance of six FMs and six MIL methods for WSI classification.
• To evaluate the impact of different feature extraction and aggregation techniques on clinical prediction tasks.
• To assess the utility of FMs in advancing MIL for pathology diagnostics.

Main Methods:

• Implemented and compared six state-of-the-art FMs (CTransPath, PathoDuet, PLIP, CONCH, UNI) as patch-level feature extractors.
• Tested various feature aggregators including attention-based pooling, transformers, and dynamic graphs.
• Evaluated performance across seven end-to-end prediction tasks on WSIs from 4044 patients with four cancer types.

Main Results:

• FMs trained on diverse datasets (e.g., UNI) outperformed generic models, improving MIL classification accuracy and convergence speed.
• Online feature re-embedding (instance feature fine-tuning) further enhanced WSI classification by capturing fine-grained details and spatial interactions.
• FMs enabled accurate WSI classification for grading, biomarker status, and MSI prediction without requiring pixel- or patch-level annotations.

Conclusions:

• Foundation models significantly advance multiple instance learning for whole slide image classification in computational pathology.
• Domain-specific FMs trained on diverse histological data offer superior performance and efficiency.
• FMs provide a powerful, annotation-free approach for critical diagnostic tasks in digital pathology.