ChatDiff: A ChatGPT-based diffusion model for long-tailed classification.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Long-tailed data distributions pose significant challenges for deep learning applications.
• Existing data augmentation methods struggle with sample diversity and negative sample interference.

Purpose of the Study:

• To introduce ChatDiff, a novel information augmentation method for improving deep learning on imbalanced datasets.
• To generate diverse positive samples for underrepresented classes and eliminate harmful negative samples.

Main Methods:

• Utilizing prompt templates to extract textual knowledge from ChatGPT-3.5 to enrich feature spaces.
• Employing a conditional diffusion model to generate semantically rich image samples for tail classes.
• Implementing a CLIP-based discriminator to filter out and remove generated negative samples.

Main Results:

• ChatDiff successfully generates diverse and semantically rich samples for underrepresented classes.
• The removal of negative samples by the CLIP discriminator prevents learning erroneous features.
• Demonstrated significant improvements in long-tailed classification performance across multiple benchmarks.

Conclusions:

• ChatDiff offers an effective solution for the long-tailed data problem in deep learning.
• The integration of large language models and diffusion models with discriminative filtering enhances data augmentation.
• Validated effectiveness on CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist 2018 datasets.