Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

HisCMCL: Cross-Modal Contrastive Learning with Hierarchical Multi-Scale Fusion for Spatial Expression Prediction.

Bioinformatics (Oxford, England)·2026
Same author

SpaCross deciphers spatial structures and corrects batch effects in multi-slice spatially resolved transcriptomics.

Communications biology·2025
Same author

SpaBatch: Deep Learning-Based Cross-Slice Integration and 3D Spatial Domain Identification in Spatial Transcriptomics.

Advanced science (Weinheim, Baden-Wurttemberg, Germany)·2025
Same author

SpaICL: image-guided curriculum strategy-based graph contrastive learning for spatial transcriptomics clustering.

Briefings in bioinformatics·2025
Same author

Improving cell-type composition inference in spatial transcriptomics with SpaDAMA.

PLoS computational biology·2025
Same author

VTrans: A VAE-Based Pre-Trained Transformer Method for Microbiome Data Analysis.

Journal of computational biology : a journal of computational molecular cell biology·2025

Related Experiment Video

Updated: Jun 9, 2025

Spatial Profiling of Protein and RNA Expression in Tissue: An Approach to Fine-Tune Virtual Microdissection
09:19

Spatial Profiling of Protein and RNA Expression in Tissue: An Approach to Fine-Tune Virtual Microdissection

Published on: July 6, 2022

4.9K

Multimodal contrastive learning for spatial gene expression prediction using histology images.

Wenwen Min1, Zhiceng Shi1, Jun Zhang1

  • 1School of Information Science and Engineering, Yunnan University, East Outer Ring Road, Chenggong District, Kunming 650500, Yunnan, China.

Briefings in Bioinformatics
|October 29, 2024
PubMed
Summary

This study introduces mclSTExp, a new AI method using H&E images and spatial transcriptomics (ST) data to predict gene expression. This cost-effective approach enhances understanding of gene patterns in complex biological systems.

Keywords:
histology imagesmultimodal contrastive learningspatial transcriptomicstransformer encoder

More Related Videos

Multimodal Hierarchical Imaging of Serial Sections for Finding Specific Cellular Targets within Large Volumes
11:19

Multimodal Hierarchical Imaging of Serial Sections for Finding Specific Cellular Targets within Large Volumes

Published on: March 20, 2018

10.4K
Quantitative Multispectral Analysis Following Fluorescent Tissue Transplant for Visualization of Cell Origins, Types, and Interactions
11:27

Quantitative Multispectral Analysis Following Fluorescent Tissue Transplant for Visualization of Cell Origins, Types, and Interactions

Published on: September 22, 2013

9.3K

Related Experiment Videos

Last Updated: Jun 9, 2025

Spatial Profiling of Protein and RNA Expression in Tissue: An Approach to Fine-Tune Virtual Microdissection
09:19

Spatial Profiling of Protein and RNA Expression in Tissue: An Approach to Fine-Tune Virtual Microdissection

Published on: July 6, 2022

4.9K
Multimodal Hierarchical Imaging of Serial Sections for Finding Specific Cellular Targets within Large Volumes
11:19

Multimodal Hierarchical Imaging of Serial Sections for Finding Specific Cellular Targets within Large Volumes

Published on: March 20, 2018

10.4K
Quantitative Multispectral Analysis Following Fluorescent Tissue Transplant for Visualization of Cell Origins, Types, and Interactions
11:27

Quantitative Multispectral Analysis Following Fluorescent Tissue Transplant for Visualization of Cell Origins, Types, and Interactions

Published on: September 22, 2013

9.3K

Area of Science:

  • Computational Biology
  • Genomics
  • Bioinformatics

Background:

  • Spatial transcriptomics (ST) offers deep insights into gene expression but is costly.
  • Hematoxylin and Eosin (H&E) whole-slide images provide an accessible alternative for gene expression prediction.
  • Current AI methods often fail to integrate multimodal H&E and ST data effectively.

Purpose of the Study:

  • To develop a cost-effective AI model for predicting spatial gene expression using H&E images and ST data.
  • To leverage multimodal information from H&E images and spatial locations for enhanced prediction accuracy.
  • To overcome the limitations of existing methods in integrating diverse data sources.

Main Methods:

  • Proposed mclSTExp, a multimodal contrastive learning framework utilizing Transformer and Densenet-121 encoders.
  • Treated spatial transcriptomics spots as 'words' to integrate intrinsic features and spatial context via Transformer self-attention.
  • Incorporated H&E image features through contrastive learning to improve predictive performance.

Main Results:

  • mclSTExp demonstrated superior performance in predicting spatial gene expression across two breast cancer and one skin squamous cell carcinoma dataset.
  • The model effectively predicted highly variable genes, showcasing enhanced predictive capabilities.
  • Achieved high accuracy in predicting spatial gene expression patterns.

Conclusions:

  • mclSTExp offers a powerful and cost-effective solution for spatial gene expression prediction.
  • The model shows promise in identifying cancer-specific genes, immune-related genes, and specialized spatial domains.
  • This approach facilitates deeper biological system analysis and disease research.