A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method.

Area of Science:

• Machine Learning
• Deep Learning
• Artificial Intelligence

Background:

• Overfitting poses a significant challenge in deep learning, degrading network generalization and performance.
• Existing regularization techniques may not fully address the complex interplay between neuron correlations and classification accuracy.

Purpose of the Study:

• To introduce a novel regularization technique, the Class-Based Decorrelation Method (CDM).
• To enhance network generalization and model accuracy by addressing neuron correlation within hidden layers.

Main Methods:

• The Class-Based Decorrelation Method (CDM) treats hidden layer neurons as base learners.
• CDM minimizes correlation among base learners while maximizing class-conditional correlation.
• The method promotes both diversity and class-specific cohesion among neurons.

Main Results:

• Experiments on various datasets using deep models show CDM's effectiveness.
• CDM significantly reduces overfitting in deep learning networks.
• Classification performance and model accuracy are demonstrably improved by CDM.

Conclusions:

• The Class-Based Decorrelation Method (CDM) is a promising regularization technique for deep learning.
• CDM offers a dual benefit of promoting neuron diversity and enhancing class-specific feature learning.
• This method effectively combats overfitting, leading to superior generalization and accuracy in deep models.