快速增强:利用语言表示来快速进行持续学习.
Wei Li1, Dezhi Li1, Shitong Shao1
1School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, Jiangsu, China.
概括
通过使用语言引导的规范化来防止遗忘,Prompt-E增强了持续的学习. 这种方法可以在不断变化的数据流上提高模型准确性,而无需显著的内存或隐私成本.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 持续学习允许模型在不忘记过去的知识的情况下顺序学习.
- 传统方法面临记忆和隐私问题;基于提示的方法提供了替代方案,但可能会丢失信息.
- 大规模的预先训练有素的视觉转换器已经在持续学习中推进了基于提示的方法.
研究的目的:
- 引入Prompt-E,一种新的方法来增强基于提示的持续学习.
- 在长期的持续学习中解决普遍的信息丢失和灾难性遗忘问题.
- 为了提高任务特定的相关性和稳定性提示跨不断变化的数据.
主要方法:
- 提示-E将语言引导的规范化纳入基于提示的持续学习中.
- 它通过使用语言表示来动态增强提示功能,以确保任务相关性.
- 该方法限制了CLS代币,并提示减轻即时冲突和遗忘.
主要成果:
- 与现有的基于提示的方法相比,Prompt-E在类增量任务中显示了显著的准确性增长.
- 它在DomainNet上的L2P精度提高了3.29%,使用最小的额外参数 (1.89M).
- 废除研究证实了语言引导的快速规范化的有效性.
结论:
- Prompt-E提供了一个轻量级且有效的解决方案,用于基于提示的持续学习.
- 提出的语言引导规范化成功地打击了灾难性的遗忘和迅速的冲突.
- 通过视觉转换器,Prompt-E代表了长期持续学习的重大进步.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
17.3K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.7K
相关概念视频
Purposive Learning
210
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
210
Language Development
462
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
462
Cognitive Learning
672
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
672
Components of Language
416
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
416
Language and Cognition
466
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
466
Introduction to Learning
551
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
551
