Rectify representation bias in vision-language models for long-tailed recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Natural data often has a long-tailed distribution, leading to poor recognition performance for rare classes.
• Existing methods primarily focus on class frequency, neglecting the crucial factor of class correlation.

Purpose of the Study:

• To investigate the performance bottleneck in long-tailed visual recognition within the visual-language (VL) framework.
• To propose a novel approach that incorporates class correlation to address recognition confusion between head and tail classes.

Main Methods:

• Modeling representation learning into special and common parts to capture unique and shared class characteristics.
• Introducing a rectification contrastive term (ReCT) to mitigate bias in common representation learning.
• Leveraging semantic hints and training status to guide the rectification process.

Main Results:

• The common representation learning is biased towards head classes, causing networks to prioritize shared features over unique ones.
• ReCT effectively rectifies representation bias, improving recognition accuracy for tail classes.
• Experiments on three long-tailed datasets show ReCT enhances performance, achieving 75.4% accuracy on iNaturalist2018 with a ResNet-50 backbone.

Conclusions:

• Class correlation is a critical factor in long-tailed visual recognition, alongside class frequency.
• The proposed ReCT method offers an effective solution for representation bias in the VL framework.
• This work advances the state-of-the-art in long-tailed recognition by improving accuracy and reducing confusion between correlated classes.