Related Experiment Videos
A novel training scheme for multilayered perceptrons to realize proper generalization and incremental learning
1Electron. and Commun. Sci. Unit, Indian Stat. Inst., Calcutta, India.
IEEE Transactions on Neural Networks
|February 2, 2008
Summary
We developed a new training method for multilayered perceptron (MLP) networks to improve classification reliability. This approach ensures MLPs do not respond to data points outside their training distribution, enhancing accuracy and enabling incremental learning.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Pattern Recognition
Background:
- Multilayered perceptron (MLP) networks often provide unreliable responses to data points far from their training set boundaries.
- Ensuring network reliability for out-of-distribution data is crucial for robust classification systems.
Purpose of the Study:
- To introduce a novel training scheme for MLPs that enhances reliability by preventing responses to data points outside the training distribution.
- To improve the handling of overlapping classes and enable incremental learning in MLPs.
Main Methods:
- A new training scheme is proposed, involving the creation and training of independent subnets for each class.
- Each subnet is trained using data from its specific class and synthetically generated points from outside the class boundary.
- A method for approximating points outside a pattern class boundary is introduced for training subnets.
Main Results:
- Simulations demonstrate that MLPs trained with the proposed method exhibit reduced responses to points outside the training sample boundary.
- The new training scheme improves the network's ability to handle overlapping classes effectively.
- The proposed method facilitates incremental training, allowing MLPs to acquire new knowledge without catastrophic forgetting.
Conclusions:
- The developed training scheme significantly enhances the reliability and robustness of MLPs as classifiers, particularly for out-of-distribution data.
- This approach offers a promising solution for improving MLP performance in complex classification tasks with overlapping classes and supports continuous learning.
Related Concept Videos
01:24Generalization, Discrimination, and Extinction
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
01:18Introduction to Learning
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
01:27Associative Learning
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
01:12Observational Learning
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning because...
01:22Purposive Learning
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a bonus...
01:19Survival Tree
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a survival tree begins...
Building a Survival Tree
Constructing a survival tree begins...