Related Experiment Video
Updated: Jul 1, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
555
CeCR: Cross-entropy contrastive replay for online class-incremental continual learning
Guanglu Sun1, Baolun Ji1, Lili Liang1
1School of Computer Science and Technology, Harbin University of Science and Technology, Harbin 150080, China.
Summary
This study introduces Cross-entropy Contrastive Replay (CeCR), a novel method for continual learning from data streams. CeCR enhances replay-based methods by improving sample selection and model training for better knowledge retention.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Computer Science
Background:
- Replay-based methods are crucial for continual learning from data streams.
- Effective sample selection from replay buffers remains a key challenge.
Purpose of the Study:
- To propose a novel method, Cross-entropy Contrastive Replay (CeCR), for online class-incremental learning.
- To address the challenge of representative sample selection in replay-based continual learning.
Main Methods:
- Class-focused Memory Retrieval for class-level sampling without replacement.
- Class-mean approximation memory update to selectively replace misclassified samples.
- Cross-entropy Contrastive Loss for robust knowledge acquisition during model training.
Main Results:
- The proposed CeCR method demonstrates comparable or improved performance on benchmark datasets.
- CeCR effectively handles the online class-incremental learning setting.
- The method achieves more solid knowledge for effective learning.
Conclusions:
- CeCR offers a promising approach for continual learning from data streams.
- The proposed sampling and memory update strategies enhance replay-based methods.
- CeCR contributes to more effective and robust incremental learning systems.
Keywords:
Buffer managementCatastrophic forgettingContrastive replayCross-entropy contrastive lossOnline class incremental learningMore Related Videos
Related Concept Videos
01:12Observational Learning
171
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
171
01:18Introduction to Learning
381
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
381
01:27Associative Learning
358
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
358
01:21Cognitive Learning
240
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
240
01:23Reinforcement
206
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
206
01:22Purposive Learning
119
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
119