Related Experiment Video
Updated: Jul 10, 2025
Author Spotlight: Impact of Intergenic Interactions on Disease-Identifying Dark Biomarkers
Published on: March 1, 2024
Transformer with convolution and graph-node co-embedding: An accurate and interpretable vision backbone for
Xiao Xiao1, Yan Kong2, Ronghan Li3
1State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China; SJTU-Yale Joint Center for Biostatistics and Data Science, National Center for Translational Medicine, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China; Department of Biostatistics, Yale School of Public Health, Yale University, New Haven, CT, United States.
TCGN, a novel method combining convolutional layers, transformers, and graph neural networks, accurately infers gene expressions from histopathological images. This approach enhances genotype-phenotype connections for precision health applications.
Area of Science:
- Computational biology
- Bioinformatics
- Digital pathology
Background:
- Inferring gene expression from histopathology images is challenging due to modality differences.
- Existing methods face issues with complexity, interpretability, and feature encoding.
Purpose of the Study:
- To develop an accurate and interpretable method for gene expression estimation from histopathological images.
- To address limitations of current approaches in model complexity and feature representation.
Main Methods:
- Developed TCGN (Transformer with Convolution and Graph-Node co-embedding method).
- Integrated convolutional layers, transformer encoders, and graph neural networks.
- Utilized single spot images as input for histopathological analysis.
Main Results:
- TCGN achieved superior performance on three spatial transcriptomic datasets (median PCC 0.232).
- The model demonstrates high accuracy with minimal parameters (86.241 million) and low memory consumption.
- TCGN offers interpretability and can be extended to bulk RNA-seq data.
Conclusions:
- TCGN is an effective tool for inferring gene expression from histopathological images.
- The method facilitates genotype-phenotype connections and biomarker prediction.
- TCGN supports precision health applications and multi-modal data modeling.
