Search research articles

Related Concept Videos

Survival Tree

Survival Tree

Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a survival tree begins...

Generalization, Discrimination, and Extinction

Generalization, Discrimination, and Extinction

Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...

Uncertainty in Measurement: Accuracy and Precision

Uncertainty in Measurement: Accuracy and Precision

Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.

Random and Systematic Errors

Random and Systematic Errors

Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...

Random and Systematic Errors

Random and Systematic Errors

Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...

Systematic Error: Methodological and Sampling Errors

Systematic Error: Methodological and Sampling Errors

In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

A feedforward bidirectional associative memory.

IEEE transactions on neural networks·2008

Same author

New nonleast-squares neural network learning algorithms for hypothesis testing.

IEEE transactions on neural networks·1995

See all related articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Videos

On overfitting, generalization, and randomly expanded training sets.

G N Karystinos¹, D A Pados

¹Department of Electrical Engineering, State University of New York, Buffalo, NY 14260-2050, USA. cary@eng.buffalo.edu

IEEE Transactions on Neural Networks

|February 6, 2008

Summary

This study introduces a novel algorithmic procedure to expand training data, enhancing the generalization of multilayer perceptrons (MLPs) by reducing overfitting. The method uses K-means clustering and Gaussian density estimation for improved model performance.

Related Experiment Videos

Area of Science:

Machine Learning
Artificial Intelligence
Pattern Recognition

Background:

Overfitting in multilayer perceptrons (MLPs) hinders generalization.
Backpropagation training requires robust datasets to avoid poor performance.

Purpose of the Study:

To develop an algorithmic procedure for random training set expansion.
To combat overfitting and enhance MLP generalization ability.

Main Methods:

K-means clustering of the training set.
Formation of locally most entropic colored Gaussian joint input-output probability density function (pdf) estimates per cluster.
Selection of the optimal number of clusters via minimum differential entropy and global cross-validation.

Main Results:

Demonstrated effectiveness of the proposed algorithmic procedure.
Improved generalization ability of backpropagation trained MLPs.
Validation through numerical studies on real and synthetic data.

Conclusions:

The developed procedure effectively expands training data to mitigate overfitting.
The method enhances the generalization capabilities of MLPs.
The approach is supported by empirical evidence from diverse datasets.