An empirical study of defect clustering in deep neural networks and its implications for testing

DNN decision logic is vulnerable to adversarial attacks, making robustness testing essential for safety-critical intelligent systems. Existing coverage-guided testing remains limited by the weak correlation between neuron coverage and defect discovery, while the feature-space distribution of vulnerable regions remains insufficiently understood. The empirical analysis investigates whether bounded perturbations reveal stable vulnerable regions in DNN feature space and whether the proximity of clean test seeds to these regions can indicate defect-revealing potential. Experiments use MNIST, CIFAR-10, and GTSRB with eight CNN models across five architectures. Vulnerability-revealing samples are generated from originally correctly classified inputs by gradient-based and boundary-based adversarial attacks, and penultimate-layer features are analyzed through PCA, HDBSCAN, UMAP, and density-based statistical tests. Metamorphic mutation testing evaluates clean seeds from different feature-space regions under the same mutation budget. The experiments are implemented in Python/PyTorch under a controlled empirical evaluation protocol. Stable and statistically significant clusters of vulnerability-revealing samples are observed across all datasets and architectures, with an average Silhouette Score of 0.538, all Davies-Bouldin Index values below 1.0, and density Z-scores ranging from 68.63 to 391.41. Seeds near defect clusters achieve an ESR of 81.7%, compared with 46.8% and 61.1% for the two baselines, and obtain 2.60/1.50 times higher defect-triggering capability and 2.72/1.76 times higher discovery efficiency.