LRNAS: Differentiable Searching for Adversarially Robust Lightweight Neural Architecture

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) require adversarial robustness for reliable deployment.
• Enhancing adversarial robustness often leads to increased network size, creating a trade-off.
• Current methods combining model compression and adversarial training depend heavily on manual neural architecture design.

Purpose of the Study:

• To propose a lightweight and robust neural architecture search (LRNAS) method.
• To automatically discover neural network architectures that are both lightweight and adversarially robust.
• To overcome the limitations of manual neural architecture design in achieving adversarial robustness and efficiency.

Main Methods:

• Developed a novel search strategy to quantify component contributions within the search space.
• Implemented a greedy strategy for architecture selection to maintain model size while incorporating beneficial components.
• Employed automated neural architecture search to identify optimal lightweight and robust models.

Main Results:

• The proposed LRNAS method successfully identified lightweight neural architectures with high natural accuracy and adversarial robustness.
• Experimental results on benchmark datasets demonstrate superiority over state-of-the-art methods against adversarial attacks.
• Ablation studies confirmed the effectiveness of individual LRNAS components in enhancing overall performance.

Conclusions:

• LRNAS effectively guarantees lightness, natural accuracy, and adversarial robustness in searched architectures.
• The method automates the design of efficient and secure deep neural networks.
• LRNAS offers a promising direction for developing practical and resilient DNNs.