Quantized Convolutional Neural Networks Robustness under Perturbation

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Machine learning models face compute limitations due to size and operations.
• Quantization reduces model size and computational needs by using lower-precision integers.
• Existing research confirms quantized models match full-precision performance but lacks analysis under input perturbations.

Purpose of the Study:

• To investigate the performance of 8-bit quantized convolutional neural networks (CNNs) under perturbed input conditions.
• To address the gap in literature regarding the robustness of quantized models in noisy environments.
• To evaluate the impact of input perturbations on model accuracy and output distribution similarity.

Main Methods:

• Studied three CNNs (ResNet-18, VGG-16, SqueezeNet1_1) in both floating-point and 8-bit quantized forms.
• Applied various noise regimes with different intensities to model inputs.
• Measured performance using top-1/top-5 accuracy, F1 score, and introduced Kullback-Liebler divergence to assess output distribution changes.

Main Results:

• Quantized models exhibited consistently low relative error compared to full-precision counterparts across all tested perturbations.
• Kullback-Liebler divergence remained comparable to unperturbed tests, indicating stable decision-making similarity.
• Significant divergences in output distributions were observed for VGG-16 and SqueezeNet1_1 specifically under Brownian noise perturbations.

Conclusions:

• 8-bit quantized CNNs demonstrate robustness and maintain performance consistency even when subjected to input noise.
• Kullback-Liebler divergence is a valuable metric for quantifying the impact of quantization on model output similarity under stress.
• While generally robust, specific models and noise types (e.g., Brownian noise) warrant further investigation for potential vulnerabilities.