The DNA dialect: a comprehensive guide to pretrained genomic language models.

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Pretrained large language models (LLMs) have advanced natural language processing and protein biology.
• Genomic language models (gLMs) are increasingly applied in genomics, leveraging diverse DNA and RNA sequence data.
• gLMs aim to enhance performance across various genomic prediction and understanding tasks.

Purpose of the Study:

• To review the evolution and current trends of genomic language models (gLMs).
• To analyze gLM components, including data curation and architecture.
• To provide researchers with an understanding of gLM capabilities, limitations, and best practices.

Main Methods:

• Review of existing literature on genomic language models.
• Analysis of gLM training data, architectures, and benchmarking efforts.
• Discussion of practical requirements and unresolved challenges for gLM implementation.

Main Results:

• The field of gLMs is rapidly evolving with increasing model complexity.
• No single gLM currently dominates; task-specific design and pretraining data are critical factors.
• gLMs show promise in applications like genome annotation and DNA sequence generation, but limitations persist.

Conclusions:

• gLMs offer significant potential for genomic research, but their effective use requires careful consideration of design and data.
• Further research is needed to address current gaps and pitfalls in gLM application.
• This review equips researchers with knowledge for the effective utilization of gLMs in genomics.