From Neuron Coverage to Steering Angle: Testing Autonomous Vehicles Effectively.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Software Engineering

Background:

• Deep Neural Networks (DNNs) in autonomous vehicles operate as complex "black boxes," necessitating methods to ensure their reliability and trustworthiness.
• Traditional software testing techniques are being adapted for DNNs, with neuron coverage proposed as a key metric for evaluating test suite effectiveness in detecting failures.

Purpose of the Study:

• To investigate the impact of realistic image transformations on neuron coverage for autonomous vehicle DNNs.
• To assess how these transformations affect the output of trained autonomous vehicle DNNs.
• To contribute to methods for increasing the reliability and trust in DNN models.

Main Methods:

• Utilized realistic image transformations to generate novel datasets for testing.
• Applied these transformations to a trained autonomous vehicle DNN.
• Measured the resulting neuron coverage and analyzed changes in model output.

Main Results:

• Realistic image transformations significantly alter neuron coverage metrics.
• The chosen transformations demonstrably impact the classification or prediction outputs of the DNN.
• Neuron coverage levels varied depending on the specific image transformation applied.

Conclusions:

• Realistic transformations are valuable for uncovering potential failures in autonomous vehicle DNNs.
• The study provides insights into the sensitivity of DNNs to input variations.
• Findings support the use of input transformation-based testing to improve DNN robustness and reliability.