Open-source framework for detecting bias and overfitting for large pathology images

Even foundational models trained on large-scale datasets may learn to rely on non-relevant artifacts such as background color or color intensity, leading to overfitting and/or bias. To ensure the robustness of deep learning applications, there is a need for methods to detect and remove the use of these artifacts. Existing debugging methods are often domain- and model-architecture-specific, and may be computationally expensive, hindering widespread use. We propose a model-architecture-agnostic framework to debug deep learning models. To demonstrate the utility of our framework, we test it using a widely used dataset from histopathology, which has been tested in other literature. The dataset features very large images that typically demand large computational resources. We demonstrate that the framework can replicate known bias patterns in a pre-trained foundation model (Phikon-v2) and a self-trained self-supervised model (MoCo v1). Our framework contributes to the development of more reliable, accurate, and generalizable models for WSI analysis, and is available as an open-source tool integrated with the MONAI framework at https://github.com/uit-hdl/feature-inspect.