Adaptive Dropout Method Based on Biological Principles.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Overfitting is a significant challenge in neural network training, often addressed by standard dropout.
• Standard dropout's random neuron activation is biologically implausible and may be suboptimal.
• The human brain's neural activation patterns offer insights for more effective regularization techniques.

Purpose of the Study:

• To propose an intelligent adaptive dropout method inspired by gene theory and brain neuron activation.
• To overcome the limitations of standard dropout's rigid and random neuron inactivation.
• To enhance neural network regularization and training efficiency.

Main Methods:

• Integration of a variational autoencoder (VAE) with existing neural networks.
• Adaptive regularization of hidden neurons by learning their zero-activity probabilities.
• Alternating iterative training to optimize neuron discarding probabilities based on weights.

Main Results:

• The proposed adaptive dropout method effectively suppresses overfitting in various neural networks.
• Demonstrated superior performance compared to standard dropout across multiple datasets.
• Achieved a reduction in the number of required neurons and improved overall training efficiency.

Conclusions:

• Adaptive dropout offers a more biologically plausible and effective approach to preventing neural network overfitting.
• This method enhances regularization by learning neuron importance, leading to better model performance.
• The technique contributes to more efficient and potentially smaller neural network architectures.