General Greedy De-Bias Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Neural networks often rely on spurious correlations, leading to poor performance on out-of-distribution (OOD) data.
• Existing de-biasing methods struggle with complex OOD scenarios or degrade in-distribution performance.

Purpose of the Study:

• To propose a General Greedy De-bias (GGD) learning framework to improve neural network robustness against spurious correlations.
• To enhance out-of-distribution generalization while maintaining in-distribution performance.

Main Methods:

• GGD greedily trains biased and base models, encouraging the base model to focus on challenging examples.
• Curriculum Regularization is introduced to balance in-distribution and OOD performance.

Main Results:

• GGD significantly improves OOD generalization across various tasks like image classification and question answering.
• The method demonstrates effectiveness with both task-specific biased models and self-ensemble approaches.
• Curriculum Regularization successfully balances in-distribution and OOD performance.

Conclusions:

• The GGD framework offers a robust solution for improving neural network generalization in the face of spurious correlations.
• GGD effectively learns a more robust base model, adaptable to different de-biasing strategies.
• The proposed Curriculum Regularization enhances the trade-off between in-distribution and out-of-distribution performance.