Benchmarking DNA Foundation Models for Genomic Sequence Classification.

Area of Science:

• Genomics
• Computational Biology
• Bioinformatics

Background:

• DNA foundation language models (FLMs) are advancing genomics by decoding DNA patterns.
• Current evaluations using fine-tuning and limited data introduce bias and limit potential assessment.

Purpose of the Study:

• To benchmark three recent DNA FLMs (DNABERT-2, NT-v2, HyenaDNA) using zero-shot embeddings.
• To evaluate model performance across diverse genomic tasks and species using 57 real datasets.
• To compare the effectiveness of mean token embedding versus sentence-level summary token embedding.

Main Methods:

• Zero-shot embedding analysis of DNABERT-2, Nucleotide Transformer version-2 (NT-v2), and HyenaDNA.
• Evaluation across 57 diverse genomic datasets and multiple species.
• Comparative analysis of mean token embedding versus sentence-level summary token embedding strategies.

Main Results:

• DNABERT-2 showed consistent performance on human genome tasks; NT-v2 excelled in epigenetic modification detection; HyenaDNA demonstrated scalability for long sequences.
• Using mean token embedding improved Area Under the Curve (AUC) by 4.3%–9.7% for all models compared to default sentence-level embeddings.
• Mean token embedding reduced performance disparities between the evaluated DNA foundation models.

Conclusions:

• Mean token embedding is a superior strategy for evaluating DNA FLMs, enhancing performance and consistency.
• The study provides a framework for selecting and optimizing DNA FLMs for genomic research.
• Findings guide researchers in the effective application of advanced language models in genomics.