Noise-enhanced convolutional neural networks.

Area of Science:

• Machine Learning
• Computer Vision
• Artificial Intelligence

Background:

• Convolutional Neural Networks (CNNs) are widely used for image recognition.
• Backpropagation is a standard training algorithm for CNNs, but can be slow.
• The generalized expectation-maximization (EM) algorithm provides a theoretical framework for understanding training dynamics.

Purpose of the Study:

• To investigate the effect of injecting carefully chosen noise into CNN training.
• To determine if noise injection can speed up the convergence of backpropagation.
• To quantify the performance improvements achieved by the proposed Noisy CNN algorithm.

Main Methods:

• The study introduces the Noisy CNN algorithm, which injects noise into the network during training.
• The algorithm leverages the relationship between backpropagation and the generalized EM algorithm.
• Noise is strategically injected, guided by a separating hyperplane in the noise space, to influence training speed.

Main Results:

• Injecting noise into output neurons reduced average per-iteration cross-entropy by 39% and classification error by 47% on the MNIST dataset.
• Noise injection can be applied to hidden layers, also yielding performance improvements.
• The benefits of noise are more pronounced with smaller datasets, as significant gains occur early in training.

Conclusions:

• Carefully chosen noise can significantly accelerate the training of CNNs.
• The Noisy CNN algorithm offers a practical method to enhance the efficiency of image recognition models.
• This approach can improve the effectiveness of random sampling from large datasets, enabling smaller samples to achieve comparable or superior results.