RNNCon: Contribution Coverage Testing for Stacked Recurrent Neural Networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Recurrent Neural Networks (RNNs) are crucial in safety-critical systems like autonomous driving.
• Existing RNN testing methods inadequately address their susceptibility to input perturbations.
• A gap exists in robust testing techniques for RNNs, threatening sequential data applications.

Purpose of the Study:

• To propose a novel test coverage metric for RNNs to enhance test adequacy and defect detection.
• To improve the performance and robustness of RNN models against input perturbations.
• To guide the generation of effective test inputs for RNNs.

Main Methods:

• Introduced RNNCon, a new coverage metric focusing on the contribution of neurons and weights within RNNs.
• Redefined contribution coverage for Stacked LSTMs and Stacked GRUs, considering joint neuron-weight effects.
• Developed and implemented RNNCon-Test, a framework prototype to guide adversarial test input generation.

Main Results:

• RNNCon covers deeper decision logic in RNNs compared to existing metrics like RNN-Test.
• RNNCon-Test successfully identified defects in Deep Learning systems.
• Adversarial inputs generated by RNNCon-Test improved model accuracy by up to 0.45% after retraining.

Conclusions:

• RNNCon provides a more detailed and effective approach to testing RNNs.
• RNNCon-Test enhances defect detection and model robustness in safety-critical sequential applications.
• The proposed metric and framework contribute to more reliable AI systems.