Generalization and selection of examples in feedforward neural networks

Area of Science:

• Machine Learning
• Computational Neuroscience
• Artificial Intelligence

Background:

• Boolean neural networks are fundamental models for studying learning.
• The choice of training data significantly impacts learning efficiency and generalization.
• Understanding generalization error is crucial for effective model training.

Purpose of the Study:

• To investigate the effect of example selection on boolean neural network learning.
• To analyze the relationship between function complexity, network architecture, and generalization.
• To develop an architecture-independent criterion for optimal training set selection.

Main Methods:

• Analytical calculation of the minimum number of examples for zero generalization error.
• Numerical simulations to validate the proposed selection criterion.
• Comparative analysis against random sampling and full dataset usage.

Main Results:

• A general, architecture-independent criterion for training example selection was proposed.
• The proposed criterion improved generalization capacity compared to random sampling across various scenarios.
• For the parity problem, specific architectures require the full dataset for global learning, a limitation addressable by tree-structured networks.

Conclusions:

• Strategic example selection is key to enhancing generalization in boolean neural networks.
• The developed criterion offers a method to optimize training data, reducing computational load.
• Network architecture plays a critical role in learning complex functions, with specific structures like tree networks offering advantages.