Multi-head ensemble of smoothed classifiers for certified robustness.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Randomized Smoothing (RS) is a key technique for certified robustness in Deep Neural Networks (DNNs).
• Ensembling multiple DNNs improves RS performance but incurs substantial computational overhead.
• Existing ensemble methods often overlook the synergistic potential and communication between individual networks.

Purpose of the Study:

• To introduce a novel, computationally efficient method for achieving certified robustness using an ensemble approach.
• To develop a training strategy that maximizes the benefits of ensembling within a single DNN architecture.
• To improve both the effectiveness and efficiency of certifiably robust defenses in DNNs.

Main Methods:

• Proposed SmOothed Multi-head Ensemble (SOME), a single DNN with multiple augmented heads trained using a cosine constraint.
• Implemented a circular communication flow and self-paced learning strategy among heads using smoothed losses.
• Designed losses specifically for certified robustness and to encourage diversity among heads.

Main Results:

• SOME demonstrates competitive or superior certifiably-robust performance compared to ensembling multiple DNNs.
• The multi-head architecture and circular-teaching scheme significantly reduce training and certification computational costs.
• Achieved a more effective and efficient defense against adversarial attacks through ensemble benefits within a single model.

Conclusions:

• The SOME approach offers a highly effective and computationally efficient solution for certified robustness in DNNs.
• The novel training strategy and architecture successfully leverage ensemble effects while mitigating computational burdens.
• SOME represents a significant advancement in developing practical and scalable certifiably robust machine learning models.