LAMBDA: A Prophage Detection Benchmark for Genomic Language Models.

Area of Science:

• Computational Biology
• Genomics
• Bioinformatics

Background:

• Transformer-based genomic sequence models are a promising area in computational biology.
• Current DNA language models (DLMs) lack the predictive power of natural and protein language models.
• Existing benchmarks for DLMs focus on eukaryotic regulatory elements, not whole-genome sequence features.

Purpose of the Study:

• To introduce LAMBDA, a novel benchmark for evaluating genome language model embeddings.
• To assess the ability of models to discriminate between phage and bacterial sequences.
• To provide a comprehensive analysis of current genomic language models.

Main Methods:

• Developed LAMBDA, a benchmark with four categories: probing tasks, fine-tuning, diagnostic tests, and genome-wide prophage detection.
• Evaluated current genomic language models using the LAMBDA benchmark.
• Analyzed the impact of training data quality and model size on performance.

Main Results:

• Genomic language models show varying performance in phage-bacteria sequence discrimination.
• Training data quality is crucial for model performance, sometimes more so than model size.
• Domain-specific training is necessary for optimal results in genomic tasks.
• Genomic language models can be applied to detect prophage sequences.

Conclusions:

• LAMBDA provides a rigorous evaluation for genome language models.
• The benchmark highlights the importance of data quality and domain-specific training for genomic language models.
• This work advances genomic annotation and addresses a key computational problem in microbiology and medicine.