BarcodeBERT: transformers for biodiversity analyses

Area of Science:

• Biodiversity research
• Genomic analysis
• Machine learning

Background:

• DNA barcodes are crucial for comparing species-specific genomic sequences.
• Existing machine learning methods for DNA barcode analysis often use generic supervised algorithms.

Purpose of the Study:

• To introduce BarcodeBERT, a specialized model for biodiversity analysis using DNA barcodes.
• To evaluate BarcodeBERT's performance against other machine learning approaches and sequence-search tools.

Main Methods:

• Developed BarcodeBERT using self-supervised pretraining on 1.5 million invertebrate DNA barcodes.
• Evaluated performance on taxonomic identification tasks, comparing against classical neural networks and general DNA foundation models.
• Analyzed masking and tokenization strategies for DNA language models.

Main Results:

• BarcodeBERT, trained on domain-specific data, outperforms fine-tuned foundation models in identifying lower taxa (genera, species).
• Achieved comparable species-level classification accuracy to BLAST but was 55x faster.
• Provided insights into optimal training strategies for customized DNA language models.

Conclusions:

• BarcodeBERT offers a significant advancement in DNA barcode analysis for biodiversity characterization.
• Self-supervised learning on domain-specific data is effective for building specialized biological language models.
• The study offers practical guidance for developing custom DNA language models tailored to specific datasets and biological domains.