Interpretable Cancer Survival Prediction by Fusing Semantic Labelling of Cell Types and Whole Slide Images.

Area of Science:

• Oncology
• Computational Biology
• Bioinformatics

Background:

• Survival prediction is a multimodal task using histopathological images and omics data.
• Genomic annotation ambiguity is a challenge in multimodal survival prediction.
• Cellular-level interpretability is crucial for understanding biological mechanisms.

Purpose of the Study:

• To develop a multimodal fusion model for accurate cancer survival prediction.
• To enhance model interpretability at cellular, gene, and histopathology image levels.
• To address genomic annotation ambiguity using cell type-specific gene annotations.

Main Methods:

• Introduced semantic annotations for genes based on cell type and cancer origin.
• Proposed SurvTransformer, a multimodal fusion model with multi-layer attention.
• Fused cell type tags (CTTs) and whole slide images (WSIs) for survival prediction.
• Utilized attention and integrated gradient attribution for interpretability.

Main Results:

• SurvTransformer achieved the highest consistency index across four cancer datasets.
• Generated statistically significant survival curves.
• Demonstrated superior performance compared to models with different labeling and attention methods.
• Case studies validated the model's interpretability at cell type, gene, and image levels.

Conclusions:

• SurvTransformer effectively integrates histopathology and genomic data for accurate survival prediction.
• The model offers novel, biologically meaningful interpretability at multiple levels.
• Cell type-specific gene annotations improve alignment with morphological features and reduce ambiguity.