Evolution and generalization of a single neurone: II. Complexity of statistical classifiers and sample size...

Area of Science:

• Machine Learning
• Statistical Classification

Background:

• Traditional approaches often treat nonlinear single-layer perceptrons (SLPs) as a distinct category.
• This research examines a broader spectrum of statistical classifiers within SLP training.

Purpose of the Study:

• To analyze the relationships between classifier complexity, generalization error, and the number of learning examples.
• To provide insights into optimizing SLP training parameters for improved performance.

Main Methods:

• Investigated seven statistical classifiers: Euclidean distance, standard Fisher linear discriminant function (DF), Fisher linear DF with pseudo-inversion, regularized linear discriminant analysis, generalized Fisher DF, minimum empirical error, and maximum margin classifier.
• Surveyed existing and novel results on classifier complexity and generalization error.

Main Results:

• Classifier performance and generalization error are contingent upon both the classifier's inherent complexity and the data's complexity.
• Identified key parameters for controlling SLP classifier complexity, including targets, learning-step, iteration count, and regularization.

Conclusions:

• Purposeful control of SLP classifier complexity through parameter tuning can reduce generalization error.
• Optimal weight initialization and data simplification further contribute to minimizing generalization error.