DeepGene: An Efficient Foundation Model for Genomics Based on Pan-Genome Graph Transformer.

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Pre-trained models like DNABERT-2 and Nucleotide Transformer have advanced DNA sequence analysis.
• Existing models struggle with genetic language diversity, model efficiency, and length extrapolation.

Purpose of the Study:

• To introduce DeepGene, a novel model designed to overcome limitations in current DNA sequence analysis.
• To enhance the capture of genetic variations and improve performance at scalable costs.

Main Methods:

• Utilized Pan-genome and Minigraph representations to capture genetic language diversity.
• Employed rotary position embedding for improved length extrapolation capabilities.
• Evaluated performance on 28 tasks within the Genome Understanding Evaluation benchmark.

Main Results:

• DeepGene achieved the overall best score on the Genome Understanding Evaluation.
• Demonstrated superior efficiency in processing DNA sequences of varying lengths.
• Outperformed other state-of-the-art models in terms of compact model size and processing speed.

Conclusions:

• DeepGene offers a significant advancement in analyzing diverse genetic language and handling variable sequence lengths.
• The model provides a more efficient and scalable solution for large-scale genetic foundational models.
• DeepGene represents a new benchmark in genome research, enhancing our ability to decode DNA sequences.