On the Robustness of Semantic Segmentation Models to Adversarial Attacks

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Deep Neural Networks (DNNs) excel at recognition tasks but are susceptible to adversarial examples.
• Semantic segmentation, a complex structured prediction task, requires specialized DNNs and has not been extensively studied for adversarial vulnerabilities.

Purpose of the Study:

• To rigorously evaluate adversarial attacks on modern semantic segmentation models.
• To analyze the impact of network architecture, model capacity, and multiscale processing on adversarial robustness in segmentation.
• To investigate how inherent model components can be leveraged for adversarial defense.

Main Methods:

• Evaluation of adversarial attacks on semantic segmentation models using two large-scale datasets.
• Analysis of different network architectures, model capacities, and multiscale processing techniques.
• Investigation of mean-field inference and input transformations for adversarial defense.

Main Results:

• Many adversarial attack observations from image classification do not directly transfer to semantic segmentation.
• Multiscale processing and mean-field inference in deep structured models naturally implement adversarial defenses.
• Established methods for benchmarking robustness and identifying inherently robust segmentation models.

Conclusions:

• Semantic segmentation models exhibit unique vulnerabilities and robustness characteristics compared to classification models.
• Existing components within deep structured models can be utilized for effective adversarial defense.
• Provides guidance for selecting robust segmentation models for safety-critical applications and future research directions in adversarial robustness.