Weight smoothing to improve network generalization

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Neural networks often struggle with generalization, especially when data patterns have inherent correlations.
• Fully-interconnected feedforward networks lack built-in mechanisms to exploit these correlations, requiring extensive training.
• Extracting neighborhood correlations is crucial for improving network performance.

Purpose of the Study:

• To propose a novel weight smoothing algorithm for neural networks.
• To enhance the generalization capability of neural networks.
• To leverage neighborhood correlations in grid-based data patterns.

Main Methods:

• Incorporating a smoothing constraint into the objective function of backpropagation.
• Applying the algorithm to fully-interconnected feedforward networks.
• Training networks on waveform classification, multifont alphanumeric character recognition, and handwritten numeral recognition tasks.

Main Results:

• Networks trained with the algorithm exhibited significantly smoother connection weights.
• The proposed algorithm led to improved generalization performance across various classification tasks.
• Demonstrated effectiveness in handling data with inherent neighborhood correlations.

Conclusions:

• The weight smoothing algorithm effectively improves neural network generalization.
• The method successfully embeds neighborhood correlation information into network training.
• This approach offers a viable solution for enhancing neural network performance on grid-based data.