Towards robust foundation models for digital pathology

Area of Science:

• Biomedical AI
• Computational Pathology
• Machine Learning in Healthcare

Background:

• Biomedical Foundation Models (FMs) are increasingly used in healthcare research and clinical validation.
• These models may learn non-biological features (e.g., lab procedures, scanner variations), posing risks for reliable clinical deployment.

Purpose of the Study:

• To introduce PathoROB, a public benchmark for quantifying the robustness of FMs to non-biological features in pathology.
• To assess both representation-level and output-level robustness of FMs in clinically relevant tasks.

Main Methods:

• Developed PathoROB benchmark for evaluating FM robustness against non-biological variations.
• Assessed representation-level robustness using a robustness index.
• Evaluated output-level robustness across patch/slide prediction, case retrieval, and clustering tasks for 20 FMs.

Main Results:

• All 20 evaluated FMs exhibited robustness deficits to non-biological features.
• Significant performance differences were observed among the evaluated FMs.
• Non-robust FM representations were found to cause major downstream diagnostic errors.

Conclusions:

• Robustness evaluation is critical for validating pathology FMs before clinical deployment.
• While robust FMs, vision-language alignment, and post-hoc methods can mitigate risks, they do not eliminate them entirely.
• PathoROB provides a framework for assessing and enhancing FM robustness in biomedical applications.