Normalizing Batch Normalization for Long-Tailed Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Real-world datasets often exhibit long-tailed distributions, where some classes have significantly more samples than others.
• Conventional deep learning models struggle with imbalanced data, leading to poor performance on under-represented (rare) classes.
• Existing methods primarily focus on data-level or classifier-level adjustments to mitigate bias.

Purpose of the Study:

• To investigate feature-level bias in deep networks trained on imbalanced data.
• To propose a novel method for rectifying feature bias by modifying Batch Normalization (BN) layer parameters.
• To enhance model performance on rare classes in long-tailed recognition tasks.

Main Methods:

• Identified that bias towards frequent classes is encoded within network features, weakening rare-class specific features.
• Introduced a parameter normalization technique for Batch Normalization (BN) layers.
• Represented BN layer Weight/Bias parameters as vectors, normalized them to unit vectors, and multiplied by a learnable scalar, decoupling parameter direction and magnitude.

Main Results:

• The proposed method effectively normalizes BN layer parameters, leading to a more balanced feature representation.
• Experiments demonstrated a significant improvement in performance on rare classes across various long-tailed benchmarks.
• The approach outperformed existing state-of-the-art methods on datasets like CIFAR-10/100-LT, ImageNet-LT, and iNaturalist 2018.

Conclusions:

• Feature bias encoded in network parameters is a key factor in poor performance on rare classes.
• Normalizing Batch Normalization (BN) layer parameters offers a simple yet effective solution to rectify feature bias.
• The proposed method presents a promising direction for improving deep learning model robustness in imbalanced data scenarios.