Benchmarking foundation models for splice site and exon annotation.

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Foundation and deep learning models have advanced genome annotation, identifying genes and splice sites.
• Current models are trained on limited datasets, potentially missing biological complexity in diverse gene elements.

Purpose of the Study:

• Evaluate foundation models for gene and splice site annotation across various exon classes.
• Assess model performance on coding, non-coding, constitutive, alternatively spliced, and transposable element-derived exons.

Main Methods:

• Evaluated transformer-based (SegmentNT, Enformer, Borzoi) and CNN-based (SpliceAI, AlphaGenome) models.
• Introduced and tested a fine-tuned CNN model (STEP2h) on specialized transposable element-exon data.

Main Results:

• Model performance was highest for exon classes present in training data (e.g., protein-coding, constitutive exons).
• Performance significantly decreased (2-4 fold) for poorly represented classes like non-coding, terminal, and alternatively spliced exons.
• Performance was also impaired on transposable element-derived exons, but a specialized fine-tuned model showed improvement.

Conclusions:

• Foundation models face challenges annotating diverse gene features not well-represented in training data.
• Further fine-tuning on specific data classes or developing task-specific models is crucial for comprehensive gene feature annotation.