Robustness to Transformations Across Categories: Is Robustness Driven by Invariant Neural Representations?

Area of Science:

• Computer Vision
• Deep Learning
• Machine Learning

Background:

• Deep convolutional neural networks (DCNNs) exhibit robustness to image transformations when trained on transformed data.
• A key hypothesis suggests DCNNs develop invariant neural representations for this robustness.
• Alternative explanations propose specialized network parts for transformed vs. non-transformed images.

Purpose of the Study:

• Investigate conditions for invariant neural representations in DCNNs.
• Determine if invariance is essential for robustness to transformations beyond training data.
• Analyze how training data composition influences invariance emergence.

Main Methods:

• Trained DCNNs with a paradigm where only specific object categories were transformed.
• Evaluated DCNN robustness to transformations on categories not seen transformed during training.
• Analyzed the emergence of invariant representations based on the proportion of transformed categories.

Main Results:

• Invariant neural representations do not consistently drive robustness; networks showed robustness for trained categories without invariance.
• Invariance emerged as the number of transformed categories in the training set increased.
• Invariance was more prominent for local transformations (blurring) than geometric transformations (rotation).

Conclusions:

• Robustness in DCNNs can be achieved without complete invariance.
• The emergence of invariance is dependent on the diversity of transformed categories in training data.
• Understanding invariance emergence is crucial for developing more robust deep learning models.