标记器选择在基因组语言模型中的影响
LeAnn M Lindsey1,2, Nicole L Pershing3, Anisa Habib1
1Kahlert School of Computing, University of Utah, SLC, UT, USA.
bioRxiv : the preprint server for biology
|August 8, 2025
概括
字符标记化最适合于核酸水平的基因组任务,优于子词方法. 字节对编码显示出对特定应用的承诺,例如SARS-CoV-2变种分类.
科学领域:
- 计算生物学是一种计算生物学.
- 基因组数据分析.
- 基因组学中的人工智能
背景情况:
- 基因组语言模型 (GLM) 是基因序列分析的新兴工具.
- GLM使用各种代码化方法,包括字符,k-mer和字节对编码 (BPE).
- 与自然语言相比,基因组序列带来了独特的挑战,影响了代币化策略.
研究的目的:
- 调查不同代币化方法对GLM业绩的影响.
- 在各种基因组分类任务中评估代币化策略.
- 在Mamba模型中,将字符标记化与子词标记化 (BPE) 进行比较.
主要方法:
- 在44个分类微调任务上评估了GLM的下游性能.
- 在Mamba状态空间模型中进行了字节对编码和字符标记化之间的直接比较.
- 根据特定任务的指标评估模型性能.
主要成果:
- 在需要核酸级别分辨率的任务上,字符标记化优于子词方法,例如拼接位置预测和促销器检测.
- 字节对编码在SARS-CoV-2变种分类中表现出卓越的性能.
- 对于大多数其他下游任务,在代币化方法之间观察到有限的统计学意义上的差异.
结论:
- 选择代币化显著影响GLM的性能,特别是在核酸水平的基因组任务.
- 字符标记化对于要求高分辨率的任务是一个强大的选择.
- 需要进一步的研究,以优化多种基因组应用和模型架构的代币化.
更多相关视频
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
682
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
579
相关概念视频
02:07Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
01:11Genetic Lingo
104.7K
Overview
104.7K
01:11Genome-wide Association Studies-GWAS
14.2K
Genome-wide association studies or GWAS are used to identify whether common SNPs are associated with certain diseases. Suppose specific SNPs are more frequently observed in individuals with a particular disease than those without the disease. In that case, those SNPs are said to be associated with the disease. Chi-square analysis is performed to check the probability of the allele likely to be associated with the disease.
GWAS does not require the identification of the target gene involved in...
GWAS does not require the identification of the target gene involved in...
14.2K
03:36Genome Annotation and Assembly
19.3K
The genome refers to all of the genetic material in an organism. It can range from a few million base pairs in microbial cells to several billion base pairs in many eukaryotic organisms. Genome assembly refers to the process of taking the DNA sequencing data and putting it all back together in a correct order to create a close representation of the original genome. This is followed by the identification of functional elements on the newly assembled genome, a process called genome annotation.
19.3K
01:22Language Development
450
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
450
02:02Genomics
37.5K
Genomics is the science of genomes: it is the study of all the genetic material of an organism. In humans, the genome consists of information carried in 23 pairs of chromosomes in the nucleus, as well as mitochondrial DNA. In genomics, both coding and non-coding DNA is sequenced and analyzed. Genomics allows a better understanding of all living things, their evolution, and their diversity. It has a myriad of uses: for example, to build phylogenetic trees, to improve productivity and...
37.5K