A modern look at simplicity bias in image classification tasks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Simplicity bias (SB) is crucial for neural network generalization but its real-world impact is unclear.
• Existing complexity measures may inaccurately assess SB, conflating it with model expressivity.

Purpose of the Study:

• To systematically investigate SB in CLIP models and its effect on image classification generalization.
• To propose a novel frequency-aware measure for quantifying SB.
• To evaluate the trade-offs of SB modulation on diverse generalization tasks.

Main Methods:

• Theoretical analysis of complexity measures.
• Development of a frequency-aware SB measure using 1-D interpolations in frequency-restricted input space.
• Modulation of SB in CLIP models for zero-shot and fine-tuning evaluations on OOD, image corruption, and adversarial tasks.

Main Results:

• The proposed measure reveals SB shifts model sensitivity towards low-frequency components, explaining generalization improvements.
• Stronger SB enhances Out-of-Distribution (OOD) generalization.
• Stronger SB correlates with decreased adversarial robustness.

Conclusions:

• SB plays a significant role in the generalization capabilities of CLIP models.
• A frequency-aware measure provides deeper insights into SB mechanisms.
• Balancing SB is critical for optimizing model performance across different generalization scenarios, particularly OOD versus adversarial robustness.