GenART: Reading the genome's language with adaptive "words"

Area of Science:

• Genomics
• Computational Biology
• Bioinformatics

Background:

• Tokenization is a key challenge for genomic language models, hindering the analysis of continuous DNA sequences.
• Existing methods often require manual annotation or struggle with biological relevance.

Purpose of the Study:

• To introduce GenART, a novel genomic language model framework.
• To develop an adaptive tokenization module for dynamic DNA segmentation.
• To improve the interpretability and performance of genomic language models.

Main Methods:

• GenART uses an adaptive tokenization module to infer biologically meaningful word boundaries from raw DNA sequences.
• The module leverages multi-scale contextual signals during pretraining.
• The framework segments DNA into variable-length words without manual annotation.

Main Results:

• GenART achieves competitive performance across various genomic tasks, surpassing existing approaches.
• Unsupervised tokenization boundaries correlate strongly with base-resolution regulatory signals like DNA methylation.
• Derived word boundaries align with annotated multi-nucleotide functional elements (GENCODE v.49).

Conclusions:

• Adaptive tokenization is an effective strategy for genomic language models.
• GenART enhances model interpretability and biological responsiveness.
• This approach facilitates a deeper understanding of DNA sequence function.