Related Experiment Videos
TriP: A triple-prompt framework aligning pre-training and class-incremental objectives in continual graph learning
Can-Ming Cui1, Hui-Yu Zhou2, Pei-Yuan Lai3
1School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, 510006, China.
Summary
We introduce TriP, a novel triple-prompt framework for Continual Graph Learning (CGL). TriP effectively mitigates catastrophic forgetting and enhances efficiency in graph-based incremental learning tasks.
Area of Science:
- Machine Learning
- Graph Neural Networks
- Artificial Intelligence
Background:
- Dynamic graph data requires Continual Graph Learning (CGL) for incremental model updates and knowledge retention.
- Class-incremental learning (class-IL) in CGL faces challenges like catastrophic forgetting and high memory overhead.
- Current prompt-based CGL methods struggle with aligning pre-training objectives and downstream tasks.
Purpose of the Study:
- To propose TriP, a triple-prompt framework for aligning pre-training and class-incremental objectives in CGL.
- To develop a lightweight, parameter isolation-based method for efficient CGL.
- To bridge the semantic gap and enhance generalization in pre-trained graph models.
Main Methods:
- Implemented a triple-prompt framework (TriP) utilizing feature-level and class-level prompts.
- Introduced a unified prompt template for aligning downstream classification with self-supervised pre-training.
- Employed parameter isolation for lightweight model design.
Main Results:
- TriP significantly outperforms state-of-the-art methods on four benchmark datasets.
- Demonstrated effectiveness in mitigating catastrophic forgetting in continual graph learning.
- Showcased improved efficiency for continual learning on graphs.
Conclusions:
- TriP offers an effective solution for class-incremental learning on dynamic graphs.
- The proposed framework successfully aligns pre-training and downstream objectives, enhancing model generalization.
- TriP presents a lightweight and efficient approach to Continual Graph Learning.
Related Concept Videos
01:12Observational Learning
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning because...
01:18Introduction to Learning
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
01:27Associative Learning
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
01:21Cognitive Learning
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
01:22Purposive Learning
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a bonus...
01:22Multi-input and Multi-variable systems
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...