Multimodal CustOmics: A unified and interpretable multi-task deep learning framework for multimodal integrative data...

Area of Science:

• Oncology
• Bioinformatics
• Computational Pathology

Background:

• Characterizing cancer requires understanding complex tumor microenvironment interactions.
• Histology images and molecular profiling offer insights but fusing these multimodal data remains challenging.
• Existing methods struggle with coherent and interpretable fusion of whole slide images and multi-omics data due to distinct biological levels and data correlations.

Purpose of the Study:

• To propose a novel deep-learning-based approach for representing multi-omics and histopathology data.
• To achieve a coherent, interpretable fusion of multimodal data for precision medicine applications.
• To develop a method robust to incomplete and missing data.

Main Methods:

• A novel deep-learning framework was developed to integrate whole slide images and multi-omics data.
• The approach extracts scores for modality activity and interactions at pathway and gene levels.
• Spatial enrichment analysis was extended for supervised tasks.

Main Results:

• The proposed method demonstrated superior performance compared to state-of-the-art approaches across multiple test cases.
• The approach effectively handles incomplete and missing data.
• Pathway activation through multimodal relationships was successfully unraveled.

Conclusions:

• The developed method offers a robust and interpretable way to fuse histopathology and multi-omics data for precision medicine.
• It provides new perspectives on understanding multimodal pathological genomic data in various cancer types.
• The method enables spatial extension of enrichment analysis and demonstrates predictive capacity.