Interpretable Multimodal Fusion Model for Bridged Histology and Genomics Survival Prediction in Pan-Cancer

Feng Gao ^1,2,3,4, Junxiang Ding ^5,6, Baowen Gai ^1,2,3

¹Department of General Surgery (Department of Colorectal Surgery), The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, 510655, China.
²Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, 510655, China.
³Biomedical Innovation Center, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, 510655, China.
⁴Shanghai Artificial Intelligence Laboratory, Shanghai, 200031, China.
⁵Guangzhou National Laboratory, Guangzhou, 510005, China.
⁶Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, 310024, China.
⁷BYHEALTH Institute of Nutrition & Health, Guangzhou, 510000, China.
⁸GZMU-GIBH Joint School of Life Sciences, The Guangdong-Hong Kong-Macau Joint Laboratory for Cell Fate Regulation and Diseases, Guangzhou Medical University, Guangzhou, 511436, China.
⁹School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, 200240, China.
¹⁰Shanghai Institute of Nutrition and Health, Chinese Academy of Sciences, Shanghai, 200030, China.
¹¹Collaborative Innovation Center for Genetics and Development, Fudan University, Shanghai, 200433, China.
¹²Shanghai Institute for Biomedical and Pharmaceutical Technologies, Shanghai, 200032, China.

⁰Department of General Surgery (Department of Colorectal Surgery), The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, 510655, China.

Advanced Science (weinheim, Baden-Wurttemberg, Germany) +

|

March 7, 2025

View abstract on PubMed

Summary

This study introduces an interpretable AI model fusing histopathology, genomics, and transcriptomics for accurate cancer prognosis. The model performs well even with missing patient data, aiding clinical decisions.

Area Of Science

Oncology
Artificial Intelligence
Medical Informatics

Background

Accurate cancer prognosis is vital for effective clinical diagnosis and treatment planning.
Multimodal artificial intelligence (AI) fusion models enhance tumor heterogeneity analysis for improved prognosis prediction.
Clinical utility of AI models is limited by incomplete multimodal patient data.

Purpose Of The Study

To develop an interpretable bridged multimodal fusion model combining histopathology, genomics, and transcriptomics.
To improve the precision of cancer patient prognosis, especially when molecular data is missing.
To create a clinically applicable AI model for cancer prognosis.

Main Methods

Developed an interpretable bridged multimodal fusion model.
Integrated histopathology, genomics, and transcriptomics data.
Validated the model across 12 cancer types with complete and missing data modalities.

Main Results

The model achieved optimal predictive performance across 12 cancer types.
Demonstrated high accuracy in both complete and missing data scenarios.
Showcased the model's utility for patients with incomplete molecular features.

Conclusions

The developed AI model offers precise cancer prognosis predictions, even with missing data.
Highlights the potential for clinically applicable multimodal fusion models in oncology.
Aims to reduce healthcare burden and enhance clinical decision-making for cancer patients.

Keywords:

artificial intelligence interpretable bridged multimodal fusion model pan‐cancer survival prediction

Related Concept Videos

Cancer Survival Analysis

Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...