Transformer for Gene Expression Modeling (T-GEM): An Interpretable Deep Learning Model for Gene Expression-Based...

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Deep learning (DL) models, often inspired by computer vision, face challenges with the unique characteristics of gene expression data in precision oncology.
• Existing DL models may lack interpretability, hindering their application in transcriptomics studies for phenotype prediction.

Purpose of the Study:

• To propose a novel interpretable deep learning architecture, Transformer for Gene Expression Modeling (T-GEM), specifically designed for gene expression data.
• To demonstrate T-GEM's capability in modeling gene-gene interactions and predicting cancer-related phenotypes.

Main Methods:

• Developed the T-GEM architecture, a Transformer-based model tailored for transcriptomics.
• Applied T-GEM to gene expression data for cancer type prediction and immune cell type classification.
• Analyzed T-GEM's attention mechanisms to understand its learning process and identify phenotype-related genes.
• Devised a method to extract gene regulatory networks learned by T-GEM using self-attention weights.

Main Results:

• T-GEM effectively models gene-gene interactions and predicts cancer-related phenotypes, including cancer type and immune cell types.
• Analysis revealed that T-GEM's attention shifts from broad gene interactions in early layers to focused, phenotype-specific genes in higher layers.
• The model's self-attention mechanism captures biologically relevant functions associated with predicted phenotypes.
• Extracted regulatory networks highlighted potential marker genes crucial for the predicted phenotypes.

Conclusions:

• T-GEM offers an interpretable deep learning approach for gene expression data analysis in precision oncology.
• The model's ability to predict phenotypes and elucidate gene regulatory networks advances the application of AI in genomics.
• T-GEM's interpretability facilitates the discovery of novel biomarkers and biological insights from transcriptomic data.