Multimodal Explainable Artificial Intelligence for Prognostic Stratification of Patients With Glioblastoma

Glioblastoma (GBM) is the most common and aggressive malignant adult tumor of the central nervous system, with a grim prognosis and heterogeneous morphologic and molecular profiles. Since the adoption of the current standard-of-care treatment in 2005, no substantial prognostic improvement has been noticed. In this study, we seek the identification of prognostically relevant GBM characteristics from routinely acquired hematoxylin and eosin-stained whole slide images (WSIs) and clinical data, which when integrated via advanced computational methods could yield improved patient prognostic stratification and hence optimize clinical decision making and patient management. The proposed WSI analysis capitalizes on a comprehensive curation of apparent artifactual content and an interpretability mechanism via a weakly supervised attention-based multiple-instance learning approach that further utilizes clustering to constrain the search space. Patterns automatically identified by our approach as of high prognostic value classify each WSI as representative of short or long survivors. Further assessments of the prognostic relevance of the associated clinical patient data are performed both in isolation and in an integrated manner, using XGBoost and SHapley Additive exPlanations. The multimodal integration of WSI with clinical data yields enhanced stratification performance when compared with using either one of the modalities. Identifying tumor morphologic and clinical patterns associated with short and long survival will enable the clinical neuropathologist to provide additional relevant prognostic information to the treating team and suggest avenues of biological investigation for further understanding and potentially treating GBM.