Spatial Omics Driven Crossmodal Pretraining Applied to Graph-based Deep Learning for Cancer Pathology Analysis.

Area of Science:

• Computational pathology
• Molecular pathology
• Artificial intelligence in medicine

Background:

• Graph-based deep learning excels in cancer histopathology by analyzing morphology and structure for outcome prediction.
• Existing methods rely on image patch embeddings as node attributes in slide graphs.
• Spatial omics, like spatial transcriptomics, offers detailed molecular information.

Approach:

• Leveraging spatial transcriptomics with contrastive crossmodal pretraining to generate deep learning models.
• Developing models to extract both molecular and histological information for graph-based learning.
• Integrating 50-micron resolution histological imaging with spatial transcriptomics data.

Key Points:

• The proposed method enhances graph-based deep learning models for histopathological slides.
• Demonstrated improvements in cancer staging, lymph node metastasis prediction, and survival prediction.
• Tissue clustering analysis also showed benefits from the integrated approach.

Conclusions:

• Spatial omics data mining significantly enhances deep learning for pathology workflows.
• Combining molecular and histological data provides a more comprehensive understanding of carcinogenesis.
• This approach shows promise for improving diagnostic and prognostic accuracy in cancer research.