Andrea Cossu1, Antonio Carta1, Lucia Passaro1

  • 1University of Pisa, Largo B. Pontecorvo, 3, Pisa, 56127, Italy.

概括

持续的预训练,在任务微调之前,模型从流数据中学习,可以减轻忘记. 在没有额外的持续学习策略的情况下,自然语言处理 (NLP) 和视觉方面的自我监督预培训被证明是有效的.

相关概念视频

Long-term Potentiation01:35

Long-term Potentiation

Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
55.1K
Language and Cognition01:27

Language and Cognition

Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
340
Forgetting01:21

Forgetting

Forgetting is an intrinsic aspect of human memory, characterized by the gradual loss or inaccessibility of information over time. Hermann Ebbinghaus, a pioneering psychologist, extensively studied this phenomenon and formulated the forgetting curve. This curve illustrates that memory loss occurs rapidly immediately after learning and then decelerates over time. Several mechanisms contribute to forgetting, including encoding failure, storage decay, retrieval failure, and interference.
Encoding...
67
Role of Cerebellum and Prefrontal Cortex in Memory01:14

Role of Cerebellum and Prefrontal Cortex in Memory

The cerebellum, while traditionally associated with motor control, also plays a crucial role in memory, particularly in procedural memory, which involves learning motor tasks that become automatic through repetition. For example, studies have shown that when the cerebellum is damaged, individuals or animals lose the ability to learn conditioned motor responses, such as the conditioned eye-blink response in classical conditioning experiments with rabbits. This study demonstrates the...
413
Interference and Decay01:16

Interference and Decay

Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
127
Implicit Memories01:24

Implicit Memories

Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
119