Quality assuring the quality assurance tool: applying safety-critical concepts to test framework development

Area of Science:

• Software Engineering
• Quality Assurance
• Embedded Systems

Background:

• Test automation is crucial for agile development and embedded systems quality.
• The quality of testing tools directly impacts the reliability of automated tests.
• Bridging agile and safety-critical development practices can enhance test framework quality assurance.

Purpose of the Study:

• To explore quality assurance approaches for test framework development in agile industries.
• To adapt insights from safety-critical system development for agile contexts.
• To identify and validate practical solutions for test framework quality assurance.

Main Methods:

• Literature review of quality assurance in functional safety and agile development.
• Analysis and synthesis of identified approaches into candidate solutions.
• Validation of candidate solutions through an industrial focus group with practitioners.

Main Results:

• Identification of 48 quality assurance approaches for test frameworks.
• Derivation of 25 candidate solutions categorized into development, analysis, run-time measures, and V&V.
• Insights such as perceiving a test framework as a tool-chain were highlighted.

Conclusions:

• The study provides actionable principles and practices for industrial test framework quality assurance.
• Candidate solutions were validated for their perceived industrial value.
• Integrating safety-critical rigor with agile flexibility offers a path to robust test frameworks.