Benchmarking foundation models as feature extractors for weakly supervised computational pathology.

Area of Science:

• Digital Pathology
• Artificial Intelligence
• Computational Biology

Background:

• Foundation models are increasingly used for extracting clinical information from histopathology images.
• Independent evaluation of these models on external datasets and tasks is limited, hindering progress.
• Benchmarking is crucial to understand model performance and identify areas for improvement.

Purpose of the Study:

• To comprehensively benchmark 19 histopathology foundation models.
• To evaluate model performance on diverse patient cohorts and cancer types (lung, colorectal, gastric, breast).
• To assess model capabilities in weakly supervised tasks including biomarker prediction, morphological analysis, and prognostic outcome determination.

Main Methods:

• Evaluated 19 foundation models on 13 patient cohorts comprising 6,818 patients and 9,528 slides.
• Utilized weakly supervised learning tasks focused on biomarkers, morphology, and prognosis.
• Compared performance of vision-language models against vision-only models.

Main Results:

• The vision-language model CONCH demonstrated the highest overall performance, followed closely by Virchow2.
• Performance advantages were less pronounced in low-data or low-prevalence scenarios.
• Foundation models trained on different cohorts learned complementary features, enabling performance enhancement through fusion.
• An ensemble of CONCH and Virchow2 outperformed individual models in 55% of tasks.
• Data diversity was found to be more critical than data volume for foundation model development.

Conclusions:

• Vision-language foundation models show superior performance in histopathology tasks compared to vision-only models.
• Ensemble methods combining complementary models can significantly improve predictive performance.
• Prioritizing data diversity over sheer volume is essential for developing robust foundation models in digital pathology.
• This study provides a benchmark for future foundation model development and evaluation in computational pathology.