Convolutional Neural Networks With Dynamic Regularization.

Area of Science:

• Machine Learning
• Deep Learning
• Computer Vision

Background:

• Overfitting is a common challenge in machine learning, particularly for convolutional neural networks (CNNs).
• Existing regularization techniques like DropBlock and Shake-Shake improve generalization but lack self-adaptive capabilities, requiring manual adjustments for different network architectures.
• Fixed regularization schedules hinder optimal performance across diverse CNN models.

Purpose of the Study:

• To propose a novel dynamic regularization method for CNNs that self-adapts its strength during training.
• To address the limitations of fixed regularization schedules in existing methods.
• To enhance the generalization performance of CNNs by dynamically balancing underfitting and overfitting.

Main Methods:

• The proposed method models regularization strength as a function of the training loss.
• The regularization strength is dynamically adjusted based on the observed changes in training loss.
• This approach allows for automatic adaptation to various network architectures and training dynamics.

Main Results:

• The dynamic regularization method demonstrated improved generalization capabilities across various off-the-shelf CNN architectures.
• Experimental results indicate that the proposed method outperforms existing state-of-the-art regularization techniques.
• The dynamic adjustment of regularization strength effectively balances underfitting and overfitting during CNN training.

Conclusions:

• The proposed dynamic regularization method offers a self-adaptive solution for improving CNN generalization.
• This approach eliminates the need for manual tuning of regularization strength, simplifying the training process.
• Dynamic regularization represents a significant advancement in mitigating overfitting and enhancing model performance in deep learning.