A clinical benchmark of public self-supervised pathology foundation models

Area of Science:

• Computational pathology and artificial intelligence in medicine.
• Development and application of foundation models in healthcare.

Background:

• Self-supervised learning for pathology foundation models has rapidly advanced.
• Recent public release of large-scale, clinically trained foundation models.
• Growing need for standardized evaluation of these models for clinical utility.

Purpose of the Study:

• To establish a benchmark for comparing the performance of public pathology foundation models.
• To assess model performance on diverse, clinically relevant tasks across multiple organs and diseases.
• To provide insights into best practices for training and selecting foundation models.

Main Methods:

• Compiled a comprehensive collection of pathology datasets from three medical centers.
• Datasets include clinical slides with endpoints such as cancer diagnoses and biomarkers.
• Systematically assessed the performance of publicly available pathology foundation models.

Main Results:

• Demonstrated the performance variations of different public pathology foundation models.
• Identified key factors influencing model performance on clinical tasks.
• Provided data-driven insights into selecting appropriate pretrained models.

Conclusions:

• The availability of public foundation models accelerates computational pathology research.
• Standardized benchmarking is crucial for bridging the gap between research and clinical deployment.
• An automated benchmarking pipeline is released to facilitate community model evaluation.