Dual-path neural network extracts tumor microenvironment information from whole slide images to predict molecular...

Utilizing AI to mine tumor microenvironment information in whole slide images (WSIs) for glioma molecular subtype and prognosis prediction is significant for treatment. Existing weakly-supervised learning frameworks based on multi-instance learning have potential in WSIs analysis, but the large number of patches from WSIs challenges the effective extraction of key local patch and neighboring patch microenvironment info. Therefore, this paper aims to develop an automatic neural network that effectively extracts tumor microenvironment information from WSIs to predict molecular typing and prognosis of glioma.

In this paper, we proposed a dual-path pathology analysis (DPPA) framework to enhance the analysis ability of WSIs for glioma diagnosis. Firstly, to mitigate the impact of redundant patches and enhance the integration of salient patch information within a multi-instance learning context, we propose a two-stage attention-based dynamic multi-instance learning network. In the network, two-stage attention and dynamic random sampling are designed to integrate diverse image patch information in pivotal regions adaptively. Secondly, to unearth the wealth of spatial context inherent in WSIs, we build a spatial relationship information quantification module. This module captures the spatial distribution of patches that encompass a variety of tissue structures, shedding light on the tumor microenvironment.

A large number of experiments on three datasets, two in-house and one public, totaling 1,795 WSIs demonstrate the encouraging performance of the DPPA, with mean area under curves of 0.94, 0.85, and 0.88 in predicting Isocitrate Dehydrogenase 1, Telomerase Reverse Tranase, and 1p/19q respectively, and a mean C-index of 0.82 in prognosis prediction. The proposed model can also group tumors in existing tumor subgroups into good and bad prognoses, with P < 0.05 on the Log-rank test.

The results of multi-center experiments demonstrate that the proposed DPPA surpasses the state-of-the-art models across multiple metrics. Through ablation experiments and survival analysis, the outstanding analytical ability of this model is further validated. Meanwhile, based on the work related to the interpretability of the model, the reliability and validity of the model have also been strongly confirmed. All source codes are released at: https://github.com/nzehang97/DPPA.