Related Experiment Video
Updated: Jun 18, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
534
Knowledge transfer in lifelong machine learning: a systematic literature review.
Pouya Khodaee1, Herna L Viktor1, Wojtek Michalowski2
1School of Electrical Engineering and Computer Science (EECS), University of Ottawa, 800 King Edward Avenue, Ottawa, ON K1N 6N5 Canada.
Artificial Intelligence Review
|July 29, 2024
Summary
Lifelong Machine Learning (LML) uses prior knowledge for new tasks. This review explores Knowledge Transfer (KT) techniques in LML, finding Parameter Isolation and Hybrid methods dominate neural network models.
Area of Science:
- Computer Science
- Artificial Intelligence
- Machine Learning
Background:
- Lifelong Machine Learning (LML) involves sequential tasks, leveraging past knowledge for new challenges.
- Knowledge Transfer (KT) is crucial for efficient adaptation and understanding in LML.
- Existing research requires a consolidated overview of current KT techniques and their application.
Purpose of the Study:
- To systematically review state-of-the-art Knowledge Transfer (KT) techniques in Lifelong Machine Learning (LML).
- To analyze evaluation metrics and datasets commonly used in LML research.
- To identify trends and future research directions in LML.
Main Methods:
- Systematic literature review of 417 articles from four databases.
- Selection of 30 highly pertinent articles for detailed analysis.
- Categorization of KT techniques into Replay, Regularization, Parameter Isolation, and Hybrid.
Main Results:
- Parameter Isolation and Hybrid techniques are predominantly used for KT in neural network (NN) frameworks.
- Supervised learning within NN models is the most common LML approach studied.
- Analysis covers both NN and non-NN frameworks, highlighting technique advantages.
Conclusions:
- The majority of LML research focuses on supervised learning with NNs.
- Future research opportunities include exploring non-NN models for Replay KT and applications beyond computer vision (CV).
- This review provides a comprehensive update for the LML research community.
More Related Videos
Related Concept Videos
01:18Introduction to Learning
357
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
357
01:21Cognitive Learning
237
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
237
01:12Observational Learning
158
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
158
02:18Types of Genetic Transfer Between Organisms
27.2K
Genetic transfer occurs when genetic information is passed from one organism to another. It occurs via two mechanisms: vertical gene transfer and horizontal gene transfer. Vertical gene transfer occurs when genetic information is transferred from one generation to the next, which happens much more frequently than horizontal gene transfer. Both sexual and asexual reproduction are forms of vertical gene transfer, where one or more organisms pass some or all of their genome onto their progeny.
27.2K
01:22Purposive Learning
107
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
107
01:35Long-term Potentiation
55.0K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
55.0K