MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) struggle with imbalanced data streams common in real-world applications.
• Existing algorithms often neglect crucial gradient information for learning from uneven class distributions.

Purpose of the Study:

• To address the challenge of learning from imbalanced data streams in DNNs.
• To leverage gradient information to improve the identification of critical patterns in minority classes.

Main Methods:

• Propose a re-weighting scheme that assigns higher weights to training samples with gradients similar to typical samples.
• Combine this re-weighting with Mixup to enhance sample space information and the re-weighting scheme's effectiveness.
• Evaluate the method, termed MixGradient, on long-tailed CIFAR and MiniPlaces data streams.

Main Results:

• MixGradient significantly boosts the generalization performance of DNNs on imbalanced data streams.
• The method achieves up to a 10% accuracy improvement across different imbalance ratios.
• Demonstrates effectiveness on challenging, artificially induced long-tailed datasets.

Conclusions:

• The proposed MixGradient method offers an effective solution for DNNs learning from imbalanced data streams.
• Highlighting important minority class samples via gradient information is crucial for improved performance.
• Combining gradient-based re-weighting with Mixup enhances learning from complex data distributions.