A multi-modal survival prediction framework with group-based batch training and structural consistency alignment

Area of Science:

• Computational pathology
• Bioinformatics
• Machine learning for healthcare

Background:

• Integrating whole-slide images (WSIs) and transcriptomic profiles is crucial for improving cancer survival prediction.
• Existing methods face challenges in balancing training efficiency and data heterogeneity due to WSI gigapixel resolution and variable sequence lengths.
• Discrepancies between histological and genomic data hinder effective cross-modal alignment and fusion, limiting prognostic accuracy.

Purpose of the Study:

• To introduce PRISM, an efficient multi-modal learning framework for integrating WSIs with transcriptomic profiles.
• To address the conflict between training efficiency and data heterogeneity preservation in existing frameworks.
• To improve cross-modal alignment and fusion for enhanced cancer survival prediction.

Main Methods:

• PRISM employs a novel stochastic partitioning strategy to divide WSIs into main and residual subsets for efficient batch training.
• It utilizes isolation masking in the main branch and a residual branch with tailored supervision to capture intra- and inter-slide correlations.
• Key modules include informative token aggregation for WSI redundancy reduction and Low-rank Bilinear Gating Fusion for efficient cross-modal interaction.

Main Results:

• PRISM achieved superior overall C-index across five TCGA cohorts compared to existing methods.
• On the TCGA-BRCA dataset, PRISM demonstrated significantly reduced training time (6 hours) compared to strong multimodal baselines.
• The framework showed the best overall Integrated Brier Score (IBS) ranking and favorable time-dependent AUC performance at 1, 3, and 5 years.

Conclusions:

• PRISM offers an effective balance between predictive performance, calibration, and computational efficiency.
• The framework shows significant potential for practical multimodal survival modeling in computational pathology.
• PRISM represents a advancement in leveraging integrated histopathology and genomics for cancer prognosis.