的架构使得快速和高效的蛋白质语言建模
Yiming Zhang1, Bian Bian2,3,4, Manabu Okumura1
1Department of Information and Communications Engineering School of Engineering, Institute of Science Tokyo Yokohama Japan.
概括
一个新的蛋白质语言模型ProtHyena使用Hyena操作符来克服变压器的限制. 这种方法在8个任务中实现了最先进的结果,减少了计算复杂性和更少的参数.
科学领域:
- 计算生物学是一种计算生物学.
- 生物信息学是一种生物信息学.
- 机器学习 机器学习
背景情况:
- 深度学习模型,特别是变形机,具有先进的生物序列分析.
- 变压器模型面临计算瓶,因为注意力机制的二次复杂性 (O(L^2)).
- 高的计算成本限制了当前模型对大型生物数据集的效率.
研究的目的:
- 介绍ProtHyena,一个高效的蛋白质语言模型.
- 为了解决基于变压器的模型在蛋白质序列分析中的计算局限性.
- 开发一个具有次方位复杂性的模型,以实现更快,更有效的记忆蛋白质建模.
主要方法:
- 使用了Hyena运算符,交替的次方位长卷曲与元素智能网关.
- 实施了一种绕过注意力机制的架构.
- 开发了一个只有160万参数的模型.
主要成果:
- 在8个不同的下游任务中实现了最先进的和可比的性能.
- 将计算复杂性显著降低到次方程级别.
- 启用了更快,更有效的记忆蛋白质序列建模.
结论:
- ProtHyena为蛋白质语言建模提供了一个高效的解决方案.
- 在生物序列分析中,Hyena操作器有效克服了变压器的局限性.
- ProtHyena为快速有效的蛋白质序列分析提供了一种有前途的方法.
更多相关视频
07:38Enabling Real-Time Compensation in Fast Photochemical Oxidations of Proteins for the Determination of Protein Topography Changes
Published on: September 1, 2020
5.7K
10:27A Fast Silver Staining Protocol Enabling Simple and Efficient Detection of SSR Markers using a Non-denaturing Polyacrylamide Gel
Published on: April 20, 2018
11.4K
相关概念视频
Language
921
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
921
Components of Language
830
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
830
Language Development
936
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
936
Language and Cognition
812
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
812
Polymer Classification: Architecture
3.9K
Polymers are classified as linear or branched on the basis of their chain architecture. The polymer chains in linear polymers have a long chain-like structure with minimal to no branching at all. Even if a polymer features large substituent groups on the monomer, which appear as branches to the skeleton, it is not considered a branched polymer. A branched polymer contains secondary polymer chains that arise from the main polymer chain. The branching occurs when the polymer growth shifts from...
3.9K
Higher Mental Functions of the Brain: Language
3.9K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.9K
