Disentangled Multi-Modal Learning of Histology and Transcriptomics for Cancer Characterization.

Area of Science:

• Computational pathology
• Bioinformatics
• Cancer research

Background:

• Histopathology is crucial for cancer diagnosis and prognosis.
• Multi-modal learning combining histology and transcriptomics offers enhanced insights.
• Existing methods face challenges with data heterogeneity, multi-scale integration, and paired data requirements.

Purpose of the Study:

• To develop a robust disentangled multi-modal framework for integrating WSIs and transcriptomics.
• To overcome limitations of existing multi-modal approaches in cancer research.
• To improve the clinical applicability and efficiency of multi-modal cancer analysis.

Main Methods:

• Decomposition of WSIs and transcriptomes into tumor and microenvironment subspaces.
• Confidence-guided gradient coordination for balanced subspace optimization.
• Inter-magnification gene-expression consistency for multi-scale integration.
• Subspace knowledge distillation for transcriptome-agnostic inference.
• Informative token aggregation for efficient WSI processing.

Main Results:

• Demonstrated superiority over state-of-the-art methods in cancer diagnosis, prognosis, and survival prediction.
• Successfully mitigated multi-modal heterogeneity and enhanced multi-scale integration.
• Enabled transcriptome-agnostic inference, reducing reliance on paired data.
• Improved inference efficiency through optimized WSI processing.

Conclusions:

• The proposed disentangled multi-modal framework significantly advances cancer multi-modal analysis.
• The framework offers a more clinically applicable and efficient approach to integrating histological and transcriptomic data.
• This work paves the way for more comprehensive and accurate cancer diagnostics and prognostics.