Single-Domain Generalization via Path Flatness-Aware Optimization of Loss Landscapes

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Traditional domain generalization (DG) requires multiple source domains.
• Single-domain DG (SDG) is more practical but challenging.
• Existing SDG methods have computational overhead and limited effectiveness.

Purpose of the Study:

• To propose a novel optimization framework for single-domain generalization.
• To address the limitations of current data augmentation and style transfer techniques in SDG.
• To enhance model robustness and cross-domain generalization capabilities.

Main Methods:

• Path Flatness-Aware Optimization (PFO) framework.
• Identifying and exploiting flat minima in the deep neural network optimization landscape.
• Iterative optimization to construct a path in parameter space for an ensemble of models.
• Initializing the optimization path using strategically interconnected model instances from anchor points determined by classification decision manifolds.

Main Results:

• PFO achieves significant performance improvements in cross-domain generalization.
• The approach implicitly aligns distributions between source and target domains within the loss landscape.
• Empirical evaluation on benchmark datasets validates the proposed method's efficacy.

Conclusions:

• Path flatness-aware optimization offers a computationally efficient and effective solution for single-domain generalization.
• The method enhances cross-domain generalization by leveraging flat minima in the optimization landscape.
• PFO provides a promising direction for improving model robustness in scenarios with limited domain data.