通过刷新学习和帕雷托优化来缓解分子性质预测中的灾难性遗忘
IEEE transactions on computational biology and bioinformatics
|August 14, 2025
概括
本研究介绍了MTL-PORL,这是一种用于大型语言模型 (LLM) 的新型持续学习 (CL) 框架,可以防止分子性质预测中的灾难性遗忘. 该方法增强了知识的保留和适应,没有先前的数据,提高了顺序学习的表现.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算化学的计算化学
背景情况:
- 持续学习 (CL) 对于大型语言模型 (LLM) 来说至关重要,以适应新的数据而不会忘记,特别是在像分子属性预测 (MP) 这样的动态领域.
- LLM经常患有灾难性遗忘 (CF),新学习会侵蚀先前的知识,特别是在化学,基因组和蛋白质组数据集中的数据分布转移时.
- 现有的基于重播的CL方法使用内存缓冲器,但忽视了插件间的关系,导致以前遇到的数据的性能不足.
研究的目的:
- 开发一种新的持续学习框架,以缓解分子性质预测的大型语言模型中的灾难性遗忘.
- 提出一个统一的等级梯度聚合框架,整合多任务学习 (MTL) 和刷新学习 (RL) 原则.
- 加强LLMs的顺序学习中的稳定性-可塑性权衡.
主要方法:
- 一个新的多任务学习框架,MTL-PORL (多任务学习者-帕雷托优化更新学习),是使用ChemBERTa模型开发的.
- 该框架包含了灵感来自神经科学的Refresh Learning (RL),它丢弃了过时的信息以改善记忆力和新的学习.
- 帕雷托优化 (PO) 和超梯度方法被用于取消学习和重新学习当前数据,以增强现有的CL方法作为插件.
主要成果:
- 在BBBP,苦和甜数据集中,MTL-PORL实现了高的随时平均准确性 (91.63%-94.89%) 和测试准确性 (92.48%-96.86%).
- 该模型显示,忘记指标很低 (-0.0048到-0.0063),表明有效的知识保留.
- 经验分析显示,与现有方法相比,顺序学习显著改善,有效地解决了稳定性-可塑性困境.
结论:
- MTL-PORL为LLM提供了持续学习的重大进步,特别是在分子性质预测任务中.
- 拟议的框架有效地打击灾难性遗忘,并改善动态数据集的性能.
- 刷新学习方法与帕雷托优化相结合,提供了一种灵活有效的解决方案,用于增强LLMs的顺序学习.
相关概念视频
Forgetting
125
Forgetting is an intrinsic aspect of human memory, characterized by the gradual loss or inaccessibility of information over time. Hermann Ebbinghaus, a pioneering psychologist, extensively studied this phenomenon and formulated the forgetting curve. This curve illustrates that memory loss occurs rapidly immediately after learning and then decelerates over time. Several mechanisms contribute to forgetting, including encoding failure, storage decay, retrieval failure, and interference.
Encoding...
Encoding...
125
Interference and Decay
203
Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
203
Woodward–Hoffmann Selection Rules and Microscopic Reversibility
3.3K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
3.3K
Predicting Reaction Outcomes
8.6K
Kinetics describes the rate and path by which a reaction occurs. In contrast, thermodynamics deals with state functions and describes the properties, behavior, and components of a system. It is not concerned with the path taken by the process and cannot address the rate at which a reaction occurs. Although it does provide information about what can happen during a reaction process, it does not describe the detailed steps of what appears on an atomic or a molecular level. On the other hand,...
8.6K
Predicting Molecular Geometry
36.0K
VSEPR Theory for Determination of Electron Pair Geometries
36.0K
Associative Learning
575
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
575


