Open-source framework for detecting bias and overfitting for large pathology images.

Area of Science:

• Artificial Intelligence
• Computational Pathology
• Medical Imaging

Background:

• Foundation models can learn biases from non-relevant image artifacts like background color.
• Existing methods to detect and remove these artifacts are often domain-specific, model-specific, and computationally expensive.
• This limits the widespread adoption of robust deep learning in medical image analysis.

Purpose of the Study:

• To develop a model-architecture-agnostic framework for debugging deep learning models.
• To address the need for reliable methods to detect and remove non-relevant artifacts.
• To improve the robustness and generalizability of models used in whole slide image (WSI) analysis.

Main Methods:

• Developed a novel, model-architecture-agnostic debugging framework.
• Tested the framework on a large-scale histopathology dataset with very large images.
• Utilized pre-trained (Phikon-v2) and self-supervised (MoCo v1) models for evaluation.

Main Results:

• The framework successfully replicated known bias patterns in both tested models.
• Demonstrated the ability to detect reliance on non-relevant artifacts like background color.
• Showcased the framework's utility in identifying model biases in WSI analysis.

Conclusions:

• The proposed framework contributes to developing more reliable and accurate deep learning models for WSI analysis.
• The model-agnostic approach enhances the generalizability of debugging methods.
• The open-source tool is integrated with the MONAI framework, facilitating wider use and development.