Delving into the adversarial robustness of semantic segmentation with decision-based black-box attacks

Semantic segmentation is critical for security-sensitive applications, yet its robustness against black-box decision-based adversarial attacks-the most practical attack setting-has not been fully investigated. In this paper, we first explore decision-based black-box attacks on semantic segmentation and clarify task-specific challenges. To address these challenges, we first propose a decision-based attack on semantic segmentation, called Discrete Linear Attack (DLA). DLA innovatively adopts discrete linear noises for perturbation exploration and calibration, combined with random search and a proxy index, to achieve high attack efficiency. We conduct adversarial robustness evaluation on 11 models from 4 datasets under 10 attacks. Specifically, DLA shows its formidable power on Cityscapes by dramatically reducing PSPNet's mIoU from an impressive 77.83% to a mere 2.14% with just 50 queries. Our code is available at https://github.com/Omenzychen/Discrete_Linear_Attack.