Predicting fine-grained cell types from histology images through cross-modal learning in spatial transcriptomics.

Area of Science:

• Computational pathology
• Spatial transcriptomics
• Single-cell biology

Background:

• Fine-grained cellular characterization is crucial for understanding tissue development, disease, and treatment response.
• Spatial cell organization impacts tumor micro-environments, heterogeneity, and prognosis.
• Current computational pathology methods have limitations in cell type identification.

Purpose of the Study:

• To develop a novel framework for identifying fine-grained cell types directly from histology images.
• To integrate morphological and molecular information for improved cell type classification.
• To enable precise analysis of cellular composition in the tumor micro-environment.

Main Methods:

• Proposed a cross-modal unified representation learning framework (CUCA).
• Trained CUCA on paired morphology-molecule spatial transcriptomics data.
• Employed cross-modal embedding alignment to harmonize image and gene expression data.

Main Results:

• CUCA successfully infers fine-grained cell types solely from pathology images.
• The model captures molecule-enhanced cross-modal representations.
• Improved prediction of fine-grained transcriptional cell abundances was achieved across three datasets.
• Downstream analyses revealed insights into tumor biology and spatial architectures.

Conclusions:

• CUCA offers a powerful tool for fine-grained cell type identification from histology images.
• The framework enhances the integration of morphological and molecular data in cancer research.
• CUCA provides novel insights into tumor spatial organization and intercellular interactions.