Transformer-based representation learning for robust gene expression modeling and cancer prognosis.

Area of Science:

• Computational Biology
• Bioinformatics
• Genomics

Background:

• Transformer models excel in NLP and vision but struggle with sparse, high-dimensional gene expression data.
• Challenges include data sparsity, high dimensionality, and missing values in transcriptomic profiles.

Purpose of the Study:

• Introduce GexBERT, a transformer-based framework for robust gene expression representation learning.
• Address limitations in applying deep learning to gene expression analysis.

Main Methods:

• GexBERT utilizes an encoder-decoder architecture pretrained on large-scale transcriptomic data.
• A masking and restoration objective captures gene co-expression relationships.
• Evaluated on pan-cancer classification, survival prediction, and missing value imputation.

Main Results:

• Achieved state-of-the-art classification accuracy using limited gene subsets.
• Enhanced cancer-specific survival prediction by restoring prognostic gene expression.
• Outperformed traditional methods in missing value imputation under high missingness.

Conclusions:

• GexBERT offers a scalable and effective tool for gene expression modeling.
• Demonstrates translational potential for gene expression analysis with limited or incomplete data.