An Empirical Investigation of Domain Generalization with Empirical Risk Minimizers.

Area of Science:

• Machine Learning
• Deep Learning
• Computer Vision

Background:

• Deep neural networks trained via Empirical Risk Minimization (ERM) show strong generalization capabilities.
• This performance often surpasses specialized algorithms in domain generalization tasks.

Purpose of the Study:

• To investigate the extent to which established domain adaptation theory explains ERM generalization.
• To identify alternative metrics that predict out-of-distribution generalization for ERM models.

Main Methods:

• Evaluated the explanatory power of Ben-David et al.'s (2007) domain adaptation theory on ERM performance.
• Analyzed various statistical measures, including Fisher information, predictive entropy, and maximum mean discrepancy, as predictors of generalization.

Main Results:

• The seminal domain adaptation theory does not tightly explain the observed out-of-domain generalization of ERM models.
• Fisher information, predictive entropy, and maximum mean discrepancy emerged as significant predictors of generalization.

Conclusions:

• Current domain adaptation theory is insufficient to fully account for ERM generalization under distribution shift.
• New theoretical frameworks are needed, potentially incorporating measures like Fisher information and predictive entropy, to understand deep network generalization.