A multimodal framework for assessing the link between pathomics, transcriptomics, and pancreatic cancer mutations.

Area of Science:

• Computational biology
• Medical image analysis
• Genomics

Background:

• Predicting genetic mutations in Pancreatic Ductal Adenocarcinoma (PDAC) from histopathology images using Deep Learning offers valuable insights.
• Integrating multi-omics data can further elucidate tumor biology mechanisms.

Purpose of the Study:

• Develop an explainable multimodal pipeline to predict KRAS, TP53, SMAD4, and CDKN2A gene mutations in PDAC.
• Integrate pathomic features from histopathology images with transcriptomics data.
• Validate the pipeline on independent datasets (TCGA-PAAD and CPTAC-PDA).

Main Methods:

• Utilized Clustering-constrained Attention Multiple Instance Learning (CLAM) models with various feature extractors (ResNet50, UNI, CONCH).
• Pre-processed RNA-seq data conventionally and with autoencoder architectures.
• Employed machine learning (ML) models for mutation classification, using attention maps and SHAP for interpretability.
• Combined pathomic and transcriptomic model outputs via a fusion layer or voting mechanism for multimodal prediction.

Main Results:

• On the validation set, multimodal ML achieved AUROC of 0.92 and AUPRC of 0.98 for KRAS mutations.
• The multimodal voting model achieved AUROC of 0.75 and AUPRC of 0.85 for TP53 mutations.
• For SMAD4 and CDKN2A, transcriptomic ML models showed AUROC of 0.71 and 0.65, respectively, with multimodal ML achieving AUPRC of 0.39 and 0.37.

Conclusions:

• The multimodal approach demonstrates potential for combining pathomics and transcriptomics.
• This interpretable framework can predict key genetic mutations in PDAC.
• The study highlights the value of integrating imaging and gene expression data for cancer research.