Class-adaptive oracle-free metamorphic test case prioritization framework for vision-based deep neural networks.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Metamorphic testing (MT) is an oracle-free strategy for evaluating deep neural networks (DNNs) using semantics-preserving perturbations.
• Existing MT prioritization methods lack class-dependent analysis and fail to integrate predictive uncertainty and interpretability drift, crucial for safety-critical vision applications.

Purpose of the Study:

• To propose a class-adaptive uncertainty-interpretability metamorphic test case prioritization framework (CUI-MTP) for vision-based deep learning systems.
• To enhance the robustness and safety-awareness of DNN validation by jointly modeling probabilistic instability and saliency-level behavioral change.

Main Methods:

• Developed a class-conditioned optimization problem for prioritizing executable metamorphic test cases.
• Integrated predictive uncertainty and interpretability drift using multi-objective Bayesian optimization with expected hypervolume improvement.
• Conducted experiments on CIFAR-10, Fashion-MNIST, and ISIC2019 datasets using ResNet and ConvNeXt architectures.

Main Results:

• The CUI-MTP framework consistently outperformed representative oracle-free baselines across various datasets, architectures, and Top-N budgets, demonstrating statistically significant improvements.
• Qualitative Grad-CAM analysis confirmed the framework's ability to identify not only prediction failures but also high-risk cases with significant saliency drift.

Conclusions:

• The proposed CUI-MTP offers a practical and scalable strategy for robust, safety-aware validation of vision-based deep learning systems.
• The framework effectively addresses limitations in existing MT prioritization by incorporating class-adaptive analysis and multi-objective optimization.