ProkBERT family: genomic language models for microbiome applications

Area of Science:

• Microbiology
• Bioinformatics
• Machine Learning

Background:

• Machine learning is vital for analyzing complex microbial data but faces challenges like data heterogeneity and limited labeled datasets.
• This study introduces ProkBERT, a novel family of large language models (LLMs) designed for genomic tasks.
• ProkBERT learns generalizable sequence representations from unlabeled genome data to overcome existing limitations.

Purpose of the Study:

• To develop a robust machine learning framework for analyzing complex microbial genomic data.
• To improve the understanding of microbial ecosystems and their impact on health and disease through advanced genomic analysis.
• To provide a generalizable sequence representation for nucleotide sequences.

Main Methods:

• ProkBERT models utilize transfer learning and self-supervised methodologies for effective microbial data analysis.
• A novel Local Context-Aware (LCA) tokenization technique is introduced to address contextual limitations of traditional transformer models.
• The methodology demonstrates adaptability across various bioinformatics tasks, retaining rich local context.

Main Results:

• ProkBERT models demonstrate superior performance in practical applications like promoter prediction and phage identification.
• Achieved high Matthews Correlation Coefficients (MCC) for promoter prediction (0.74 in E. coli, 0.62 in mixed-species) and phage identification (0.85).
• Consistently outperformed established tools in phage identification, showcasing exceptional accuracy and generalizability.

Conclusions:

• The ProkBERT model family offers a compact yet powerful tool for rapid and accurate analyses in microbiology and bioinformatics.
• Its adaptability across diverse tasks represents a significant advancement in machine learning applications within microbiology.
• ProkBERT models are publicly available on GitHub and HuggingFace, providing an accessible resource for the scientific community.