Three-Dimensional Reconstruction Pre-Training as a Prior to Improve Robustness to Adversarial Attacks and Spurious...

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Image classifiers struggle with adversarial attacks and spurious correlations.
• Adversarial training with adversarial examples is a key robustness method.
• Human vision models offer insights into structured priors for image formation.

Purpose of the Study:

• To explore a synthesis of adversarial training and 3D geometry priors for improved image classifier robustness.
• To investigate the impact of 3D reconstruction pre-training on adversarial robustness and spurious correlation resistance.

Main Methods:

• Combined adversarial training with weight initialization encoding 3D object priors via 3D reconstruction pre-training.
• Introduced the Geon3D dataset for systematic evaluation of 3D pre-training effects.
• Evaluated performance on Geon3D and ShapeNet datasets, comparing with alternative pre-training protocols.

Main Results:

• 3D reconstruction pre-training improved adversarial training robustness in realistic conditions (textured backgrounds, ShapeNet).
• The approach enhanced robustness against spurious correlations between shape and background textures.
• 3D-based pre-training outperformed 2D-based pre-training on the ShapeNet dataset.

Conclusions:

• Structured, 3D-based models of vision can significantly benefit adversarial robustness.
• Integrating 3D geometry priors offers a promising direction for developing more resilient image classifiers.
• Further research into 3D-based vision models for adversarial robustness is encouraged.