在语境学习可以扭曲序列概率和生物适应性之间的关系
Pranav Kantroo1,2, Günter P Wagner3,4,5, Benjamin B Machta6,2
1Computational Biology and Bioinformatics Program, Yale University, New Haven, CT-06520, USA.
ArXiv
|May 2, 2025
概括
语言模型可以预测生物序列的可行性. 在上下文学习可以扭曲这些预测,特别是对于重复动机的序列,通过导致异常高的概率得分.
科学领域:
- 计算生物学 计算生物学
- 生物信息学是一种生物信息学.
- 机器学习 机器学习
背景情况:
- 语言模型 (LMs) 越来越多地用于预测生物序列适应性.
- 这些模型学习序列语法,将更高的概率分数分配给可行的序列.
研究的目的:
- 调查语境学习如何影响序列概率和适应性之间的关系.
- 确定蛋白质语言模型中这种扭曲背后的机制.
主要方法:
- 使用了各种蛋白质语言模型,经过面具语言建模的训练.
- 分析了包含重复动机的序列的概率得分,包括不完美的重复.
- 检查了具有生物学相关特征的序列,例如RNA毛结构.
主要成果:
- 在上下文学习扭曲了适应性-概率相关性,特别是在具有重复动机的序列中.
- 基于变压器的模型非常容易受到这种影响,原因是检索机制.
- 这种现象也会影响不完美的重复和其他序列特征,如RNA针头图案.
结论:
- 在语境学习可以导致在生物序列模型中误导性高的概率得分.
- 变压器模型中的检索行为可以取代已学到的生物先验.
- 仔细考虑上下文学习效应对于准确的生物序列分析至关重要.
相关概念视频
Gene Evolution - Fast or Slow?
7.0K
The genomes of eukaryotes are punctuated by long stretches of sequence which do not code for proteins or RNAs. Although some of these regions do contain crucial regulatory sequences, the vast majority of this DNA serves no known function. Typically, these regions of the genome are the ones in which the fastest change, in evolutionary terms, is observed, because there is typically little to no selection pressure acting on these regions to preserve their sequences.
In contrast, regions which code...
In contrast, regions which code...
7.0K
Frequency-dependent Selection
21.6K
When the fitness of a trait is influenced by how common it is (i.e., its frequency) relative to different traits within a population, this is referred to as frequency-dependent selection. Frequency-dependent selection may occur between species or within a single species. This type of selection can either be positive—with more common phenotypes having higher fitness—or negative, with rarer phenotypes conferring increased fitness.
21.6K
Inclusive Fitness
35.8K
Most altruistic behavior—in which one animal helps another at a cost to themselves—occurs between relatives. Scientists think these altruistic behaviors evolved because they increase the inclusive fitness of the animal providing help.
35.8K
Multi-species Conserved Sequences
3.9K
Next-generation sequencing technologies have created large genomic databases of a variety of animals and plants. Ever since the human genome project was completed, scientists studied the genome of primates, mammals, and other phylogenetically distant living beings. Such large-scale studies have provided new insights into the evolutionary relationship between organisms.
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
3.9K
What is Natural Selection?
113.7K
Natural selection is an evolutionary process in which individuals with survival-promoting traits reproduce at higher rates. These favorable traits become more common within a population or species. Naturally selected traits initially arise via random genetic mutations. In order for selection to occur, there must be variation within a population, the trait controlling the variation must be heritable, and there must be an evolutionary advantage for variation in the trait.
113.7K
Cognitive Learning
96
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
96


