Probabilistic Contrastive Learning for Long-Tailed Visual Recognition.

Area of Science:

• Machine Learning
• Computer Vision
• Data Science

Background:

• Real-world data often exhibits long-tailed distributions, with numerous minority classes having few samples.
• This imbalance significantly degrades standard supervised learning algorithms.
• Supervised contrastive learning shows promise for imbalance but requires large batches, which are difficult with imbalanced data.

Purpose of the Study:

• To propose a novel probabilistic contrastive (ProCo) learning algorithm to overcome limitations of supervised contrastive learning on imbalanced datasets.
• To enable effective contrastive learning with smaller batches by estimating class feature distributions.
• To improve performance on tasks affected by data imbalance.

Main Methods:

• Proposed a probabilistic contrastive (ProCo) learning algorithm.
• Assumed normalized features follow a mixture of von Mises-Fisher (vMF) distributions.
• Estimated distribution parameters using the first sample moment for online computation.
• Sampled contrastive pairs from estimated distributions and derived expected contrastive loss for optimization.
• Extended ProCo to semi-supervised learning by generating pseudo-labels.

Main Results:

• ProCo successfully addresses the challenge of constructing contrastive pairs in imbalanced datasets.
• The use of vMF distributions allows efficient estimation and sampling of contrastive pairs.
• ProCo demonstrated consistent outperformance over existing methods in supervised and semi-supervised visual recognition and object detection tasks.
• Theoretical analysis of ProCo's error bound was conducted.

Conclusions:

• Probabilistic contrastive (ProCo) learning offers a robust solution for data imbalance in machine learning.
• The proposed method effectively utilizes feature distribution estimation for improved contrastive learning.
• ProCo shows broad applicability and superior performance across various computer vision tasks.