中型蛋白质语言模型在现实数据集上的转移学习方面表现良好
Luiz C Vieira1, Morgan L Handojo1, Claus O Wilke2
1Department of Integrative Biology, The University of Texas at Austin, Austin, TX, USA.
Scientific reports
|July 2, 2025
概括
较大的蛋白质语言模型 (pLM) 不总是更好的转移学习. 像ESM C 600M这样的中型模型具有平均嵌入式,提供了性能和效率的实际平衡.
科学领域:
- 计算生物学是一种计算生物学.
- 生物信息学是一种生物信息学.
- 在蛋白质科学中的机器学习
背景情况:
- 蛋白质语言模型 (pLMs) 提供了对蛋白质进化和结构的洞察.
- 像ESM-2 (15B参数) 这样的大型pLM捕获复杂的模式,但计算密集.
- 模型大小会影响转移学习应用中的性能.
研究的目的:
- 系统地评估蛋白质语言模型大小对转移学习表现的影响.
- 为了比较不同的嵌入式压缩方法用于特征提取.
- 为了确定最佳的pLM配置,平衡性能和计算成本.
主要方法:
- 在各种生物数据集中对各种ESM样式的蛋白质语言模型进行系统评估.
- 从预先训练的pLM中使用特征提取来转移学习.
- 嵌入式压缩技术的比较,包括平均嵌入式.
主要成果:
- 较大的蛋白质语言模型并不总是优于较小的蛋白质语言模型,尤其是在有限的数据的情况下.
- 中型模型 (ESM-2 650M,ESM C 600M) 的表现强,略低于大型模型.
- 平均嵌入性能优于其他压缩方法用于转移学习.
结论:
- 与平均嵌入相结合的ESM C 600M在性能和效率之间提供了最佳的权衡.
- 这种组合是生物应用中转移学习的实用和可扩展的解决方案.
- 模型大小并不是基于plm的转移学习成功的唯一决定因素.
相关概念视频
Improving Translational Accuracy
2.7K
2.7K
Typical Model Studies
444
Fluid mechanics model studies often utilize scaled-down systems to predict fluid behavior in full-scale environments, such as river flows, dam spillways, and structures interacting with open surfaces. Maintaining Froude number similarity in river models is crucial, as it replicates surface flow features like wave patterns and velocities.
444
Introduction to Learning
545
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
545
Observational Learning
319
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
319
Associative Learning
597
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
597
Generalization, Discrimination, and Extinction
816
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
816


