On Connections Between Regularizations for Improving DNN Robustness.

Area of Science:

• Machine Learning
• Deep Learning Theory
• Computer Vision

Background:

• Deep neural networks (DNNs) are susceptible to adversarial attacks.
• Improving adversarial robustness is crucial for reliable AI systems.
• Various regularization techniques have been proposed to enhance DNN robustness.

Purpose of the Study:

• To theoretically analyze recent regularization terms for improving DNN adversarial robustness.
• To investigate the connections between different regularization methods.
• To re-interpret the functionality of these regularizations for DNNs with rectified linear activations.

Main Methods:

• Theoretical analysis of regularization terms.
• Investigating connections between input-gradient regularization, Jacobian regularization, curvature regularization, and cross-Lipschitz functional.
• Studying DNNs with general rectified linear activations.

Main Results:

• Established theoretical connections between several effective adversarial robustness regularization methods.
• Provided new interpretations of the functionality of these regularization techniques.
• Highlighted essential components of current regularization approaches.

Conclusions:

• The theoretical insights gained can guide the development of more principled and efficient regularization methods.
• Future research can build upon these findings to create more robust deep learning models.
• Understanding the interplay of different regularization techniques is key to advancing adversarial robustness.