标记器选择在基因组语言模型中的影响
LeAnn M Lindsey1,2, Nicole L Pershing3, Anisa Habib1
1Kahlert School of Computing, University of Utah, UT, USA.
Bioinformatics (Oxford, England)
|August 18, 2025
概括
基因组语言模型从核酸水平任务的字符标记化中受益,例如拼接位置预测. 字节对编码对特定应用具有优势,但字符标记化在各种基因组任务中通常表现更好.
科学领域:
- 计算生物学 计算生物学
- 基因组学就是基因组学.
- 机器学习 机器学习
背景情况:
- 基因组语言模型 (GLMs) 为遗传序列分析提供了新的方法.
- GLM 采用了多种不同的令牌化策略,包括字符,k-mer 和字节对编码 (BPE).
- 与自然或蛋白质语言相比,基因组数据在代币化方面存在独特的挑战,原因是其低可变性和复杂的特征.
研究的目的:
- 调查不同代币化策略对GLM业绩的影响.
- 为了比较字符标记化与GLM中的BPE等子词方法.
- 评估多种下游基因组分类任务中的代币化效应.
主要方法:
- 使用44个分类微调任务评估GLM的下游性能.
- 在Mamba状态空间模型中进行了字节对编码和字符标记化的直接比较.
- 在需要核酸级别分辨率和更广泛的基因组分类的任务上使用基准标记化方法.
主要成果:
- 字符标记化在核酸特定任务上表现优于子词方法,例如拼接位置预测和促进器检测.
- 字节对编码证明了SARS-CoV-2变种分类的卓越性能.
- 对于大多数其他下游任务,在代币化方法之间观察到有限的统计学意义上的差异.
结论:
- 字符标记化对于要求核酸水平精度的基因组任务非常有效.
- 在GLM中选择代币化方法时,应以下游应用的具体要求为指导.
- 对各种基因组数据类型的最佳代币化进行进一步的研究是有必要的.
更多相关视频
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
681
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
575
相关概念视频
02:07Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
01:11Genetic Lingo
104.6K
Overview
104.6K
01:11Genome-wide Association Studies-GWAS
14.1K
Genome-wide association studies or GWAS are used to identify whether common SNPs are associated with certain diseases. Suppose specific SNPs are more frequently observed in individuals with a particular disease than those without the disease. In that case, those SNPs are said to be associated with the disease. Chi-square analysis is performed to check the probability of the allele likely to be associated with the disease.
GWAS does not require the identification of the target gene involved in...
GWAS does not require the identification of the target gene involved in...
14.1K
03:36Genome Annotation and Assembly
19.3K
The genome refers to all of the genetic material in an organism. It can range from a few million base pairs in microbial cells to several billion base pairs in many eukaryotic organisms. Genome assembly refers to the process of taking the DNA sequencing data and putting it all back together in a correct order to create a close representation of the original genome. This is followed by the identification of functional elements on the newly assembled genome, a process called genome annotation.
19.3K
01:22Language Development
447
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
447
02:02Genomics
37.4K
Genomics is the science of genomes: it is the study of all the genetic material of an organism. In humans, the genome consists of information carried in 23 pairs of chromosomes in the nucleus, as well as mitochondrial DNA. In genomics, both coding and non-coding DNA is sequenced and analyzed. Genomics allows a better understanding of all living things, their evolution, and their diversity. It has a myriad of uses: for example, to build phylogenetic trees, to improve productivity and...
37.4K