An Information-Theoretic Method to Automatic Shortcut Avoidance and Domain Generalization for Dense Prediction Tasks

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Deep convolutional neural networks (CNNs) for dense prediction tasks often use synthetic data due to annotation costs.
• Models trained on synthetic data exhibit poor synthetic-to-real (S2R) generalization, limiting their real-world applicability.
• Shortcut learning, driven by synthetic data artifacts, is identified as a key factor in this generalization gap.

Purpose of the Study:

• To address the poor S2R generalization of CNNs trained on synthetic data.
• To mitigate the influence of synthetic data artifacts on feature representation learning.
• To develop a method for learning robust and shortcut-invariant features.

Main Methods:

• Proposed an Information-Theoretic Shortcut Avoidance (ITSA) approach to restrict shortcut-related information in feature representations.
• Minimized the sensitivity of latent features to input variations to regularize robust feature learning.
• Developed a practical algorithm to achieve robustness, avoiding high computational costs of direct input sensitivity optimization.

Main Results:

• The proposed method significantly improved S2R generalization across diverse dense prediction tasks, including stereo matching, optical flow, and semantic segmentation.
• ITSA enhanced the robustness of synthetically trained networks.
• The method outperformed models fine-tuned on real data in challenging out-of-domain applications.

Conclusions:

• The ITSA approach effectively overcomes shortcut learning in synthetic data for dense prediction tasks.
• This method enhances the practical utility of synthetically trained models in real-world scenarios.
• ITSA offers a robust alternative to traditional fine-tuning for improving S2R generalization.