Co-assistant networks by pathology foundation model and convolutional neural network for gigapixel whole slide image...

Multiple instance learning (MIL) with pre-trained models to extract patch-level features has been widely used in whole slide image (WSI) analysis to avoid expensive pixel-level annotations. Although pre-trained pathology foundation model (PFM) have achieved promising performance on WSI analysis, their performance is still restricted by two key challenges: (i) self-attention mechanisms might encode trivial or noisy relations during fine-grained feature aggregation, and (ii) self-attention mechanisms struggle to capture local patterns. To overcome these limitations, we propose an Interpretable Large-Small Co-assistant (ILSC) framework, which synergistically integrates a PFM with a small convolutional neural network (CNN) to leverage their complementary advantages. The framework comprises three core components: (i) a general-feature extraction model that leverages a pre-trained PFM with adapter and attention modules to capture global and universal pathological features, (ii) a specific-feature extraction model that employs a CNN with cell-level attention to mine discriminative task-specific local features, and (iii) a feature fusion module that integrates both pathways using patch-attention for slide-level classification. Extensive experiments demonstrate that the proposed framework achieves superior classification performance compared to recent state-of-the-art methods, while also offering enhanced interpretability and generalizability. Furthermore, experiments illustrate that the small CNN model can boost the interpretability of PFM, while the pre-trained PFM can strengthen the generalizability of CNN for WSI analysis. All source codes are available athttps://github.com/lZhuoRan/ILSC.