A systematic study of the class imbalance problem in convolutional neural networks

Area of Science:

• Deep Learning
• Machine Learning
• Computer Vision

Background:

• Class imbalance is a prevalent challenge in machine learning.
• Limited systematic research exists on class imbalance in deep learning contexts.
• Convolutional Neural Networks (CNNs) are susceptible to performance degradation due to imbalanced data.

Purpose of the Study:

• To systematically investigate the impact of class imbalance on CNN classification performance.
• To compare various methods for mitigating class imbalance in deep learning.
• To evaluate the effectiveness of oversampling, undersampling, two-phase training, and thresholding.

Main Methods:

• Experiments conducted on three benchmark datasets: MNIST, CIFAR-10, and ImageNet.
• Evaluation metric: Area Under the Receiver Operating Characteristic Curve (ROC AUC) adapted for multi-class problems.
• Comparison of oversampling, undersampling, two-phase training, and thresholding techniques.

Main Results:

• Class imbalance significantly degrades CNN classification performance.
• Oversampling consistently outperformed other methods across various scenarios.
• Optimal oversampling eliminates imbalance; optimal undersampling ratio varies.
• Oversampling does not induce overfitting in CNNs, unlike in some classical models.
• Thresholding is effective for improving overall classification counts when considering prior probabilities.

Conclusions:

• Class imbalance poses a significant challenge for CNNs.
• Oversampling is the recommended strategy for addressing class imbalance in CNNs.
• Thresholding can be a useful supplementary technique for specific evaluation goals.