HisCMCL: Cross-Modal Contrastive Learning with Hierarchical Multi-Scale Fusion for Spatial Expression Prediction

Area of Science:

• Computational Biology
• Bioinformatics
• Medical Imaging Analysis

Background:

• High costs and complexity limit clinical use of spatial transcriptomics (ST).
• Inferring ST from pathology images is a promising alternative but faces challenges in feature alignment.
• Existing models lack multi-scale and cross-sample feature extraction, hindering generalization.

Purpose of the Study:

• To develop a novel multimodal framework for accurate spatial transcriptomics inference from pathology images.
• To address the limitations of single-scale modeling and improve feature alignment between image and gene expression data.
• To enhance the clinical applicability of spatial transcriptomics through efficient inference methods.

Main Methods:

• Proposed HisCMCL, a multimodal framework integrating multi-scale features and cross-attention.
• Fused spatial location information and employed contrastive learning for effective image and transcriptomic feature alignment.
• Evaluated the model on four public datasets to assess predictive performance.

Main Results:

• HisCMCL significantly outperforms existing baseline methods in predictive performance for spatial transcriptomics inference.
• The model demonstrates good structural consistency in identifying cancer and immune markers.
• HisCMCL effectively delineates tumor regions, providing new insights for spatial expression inference.

Conclusions:

• HisCMCL offers an effective solution for inferring spatial transcriptomics from pathology images.
• The framework's multi-scale and contrastive learning approach improves feature alignment and model generalization.
• HisCMCL shows potential for advancing cancer research and clinical diagnostics through enhanced spatial expression analysis.