Learning domain invariant representations by joint Wasserstein distance minimization.

Area of Science:

• Machine Learning
• Computer Science
• Statistics

Background:

• Domain shifts in training data are prevalent in real-world machine learning applications.
• Standard machine learning losses lack guarantees for consistent performance across diverse data domains.
• Ensuring domain-invariant representations is crucial for robust model generalization.

Purpose of the Study:

• To establish new theoretical foundations for addressing domain shifts in machine learning.
• To mathematically relate classical supervised machine learning losses to the Wasserstein distance.
• To develop methods for improving domain invariance and prediction stability.

Main Methods:

• Developed theoretical relationships between supervised machine learning losses and Wasserstein distance in joint space.
• Integrated a Generative Adversarial Network (GAN)-type discriminator to enforce domain invariance.
• Utilized classification and regression losses in conjunction with the discriminator.

Main Results:

• Demonstrated that combined losses and GAN-type discriminator provide an upper bound to the true Wasserstein distance between domains.
• Achieved more invariant representations and stable prediction performance across domains.
• Empirically validated theoretical results on multiple image datasets.

Conclusions:

• The proposed approach enhances domain invariance, leading to more stable prediction performance.
• The method systematically yields higher minimum classification accuracy across domains.
• This work provides a theoretical and empirical basis for building more robust machine learning models against domain shifts.