Investigating the performance of foundation models on human 3'UTR sequences.

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Foundation models (e.g., DNABERT, Nucleotide Transformer) are powerful tools in DNA research, trained unsupervisedly on large genomic datasets.
• Current models often overlook genome partitioning, training and evaluating on entire genomes rather than specific functional regions.

Purpose of the Study:

• To investigate the effectiveness of genome-wide versus 3' untranslated region (3'UTR)-specific foundation models on human 3'UTR regions.
• To evaluate these models on downstream tasks relevant to RNA biology.

Main Methods:

• Trained transformer architectures on a 3'UTR-specific dataset (3,783,714 sequences, 6.6B bp) from 241 Zoonomia species.
• Evaluated models on RNA-binding protein motif recognition, functional genetic variant detection, expression level prediction (MPRA), and mRNA half-life estimation.

Main Results:

• 3'UTR-specific foundation models showed superior performance compared to genome-wide models in three out of four evaluated downstream tasks.
• The study demonstrates significant advantages of specialized training data for specific genomic regions.

Conclusions:

• Training foundation models on functionally distinct genomic regions, such as 3'UTRs, is crucial for optimizing performance in downstream biological applications.
• The developed 3'UTR-specific tasks provide a valuable benchmark for future foundation model development in RNA biology.