MoHD: Multi-mOdal survival prediction through Hierarchical Decoupling of whole-slide image pyramids and genomics

Integrative analysis of complementary phenotype information from multi-modality data, such as pathological images and genomic profiles, has shown significant value in cancer survival prediction. However, multimodal survival prediction confronts two challenges: (1) the consistency and specificity of multimodal data remains underexplored, leading to incomplete information utilization and redundancy caused by overlapping information across modalities; (2) the correlation between inherent hierarchical structure of histopathological Whole Slide Images (WSIs) and genomic profiles has not been fully modeled. To address these issues, in this paper, we propose a Multi-mOdal survival prediction framework through Hierarchical Decoupling of whole-slide image pyramids and genomics (MoHD). Our MoHD incorporates adversarial information decomposition and hierarchical cross-modal interaction to advance survival prediction performance, which consists two core components: (i) a Multi-Granularity Feature Optimizer (MGFO) employing adversarial decoupling strategy to extract modality-common and refine modality-specific features while implementing redundancy suppression; (ii) a Multimodal Hierarchical INteractor (MHIN) that sufficiently captures multi-resolution cross-modal correlations and effectively integrates consistent and specific information through two scale-oriented interactors. We conduct extensive experiments on five cancer cohorts from the Cancer Genome Atlas (TCGA) database. The experimental results demonstrate that the proposed method achieves the superior performance compared to both unimodal and multi-modal survival prediction methods.