A generalizable pathology foundation model using a unified knowledge distillation pretraining framework

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Foundation models

Background:

• Generalization is critical for clinical adoption of foundation models in computational pathology (CPath).
• Current models are evaluated on limited tasks, hindering assessment of broad clinical applicability.

Purpose of the Study:

• To establish a comprehensive benchmark for evaluating foundation model generalization in CPath.
• To develop an improved foundation model with enhanced generalization capabilities for CPath tasks.

Main Methods:

• A benchmark comprising six clinical task types and 72 specific tasks was created.
• A unified knowledge distillation framework, incorporating expert and self-knowledge distillation, was proposed.
• The Generalizable Pathology Foundation Model (GPFM) was developed based on this framework.

Main Results:

• Existing foundation models demonstrate variable performance across different task types.
• GPFM achieved an average rank of 1.6 on the benchmark, ranking first in 42 out of 72 tasks.
• The proposed framework effectively enhances image representation learning.

Conclusions:

• Foundation models require further development to generalize effectively across the spectrum of CPath clinical tasks.
• GPFM shows significant promise as a generalized feature representation method for CPath.
• Knowledge distillation is a viable strategy for improving pathology foundation model generalization.