Advancing bioinformatics with language models: components, applications, and perspectives.

Area of Science:

• Bioinformatics and Computational Biology
• Artificial Intelligence and Machine Learning

Background:

• Large language models (LLMs) are advanced deep learning models excelling in natural language processing.
• LLMs, with billions of parameters trained on vast datasets, offer significant potential beyond text analysis.
• Their application in bioinformatics is rapidly expanding, addressing complex biological data challenges.

Purpose of the Study:

• To provide a comprehensive review of transformer-based LLM applications in key bioinformatics domains.
• To discuss essential components of LLMs for biological data, including tokenization, architectures, and pretraining.
• To survey current foundation models and their uses, while identifying future research directions.

Main Methods:

• Review of transformer-based model applications across genomics, transcriptomics, proteomics, drug discovery, and single-cell analysis.
• Discussion of critical LLM components: tokenization for biological data, transformer architectures, attention mechanisms, and pretraining strategies.
• Survey of existing foundation models and their downstream bioinformatics applications.

Main Results:

• LLMs are successfully applied in diverse bioinformatics fields, including genomics and drug discovery.
• Key components like specialized tokenization and pretraining are crucial for effective biological data processing.
• A range of foundation models are available, with numerous downstream applications demonstrated.

Conclusions:

• Transformer-based LLMs offer powerful tools for tackling complex bioinformatics problems.
• Addressing current challenges and focusing on next-generation biological language models will drive future advancements.
• Practical guidance for users and developers is provided to facilitate LLM adoption in biology.