Technology adoption performance evaluation applied to testing industrial REST APIs

Area of Science:

• Software Engineering
• Artificial Intelligence in Software Testing
• Automated Software Testing

Background:

• Manual software testing is labor-intensive and prone to errors.
• Automated test-case generation offers a solution to improve efficiency and correctness.
• Various AI (e.g., evolutionary, LLM) and algorithmic approaches exist, each with unique strengths and weaknesses for industrial integration.

Purpose of the Study:

• To present a systematic approach for evaluating test-case generation methodologies.
• To integrate these methodologies into a scalable enterprise setup for IT product development.
• To specifically address black-box testing of REST APIs using OpenAPI schemas.

Main Methods:

• Developed and applied the Technology Adoption Performance Evaluation (TAPE) approach.
• Conducted a case study within Volkswagen AG's Group IT.
• Evaluated existing tools: OpenAPI Generator, EvoMaster, and StarCoder, based on diverse technologies.

Main Results:

• The TAPE approach provides a framework for assessing test-case generation tools.
• Evaluated tools demonstrated significant benefits for test engineers.
• Facilitated test-case specification and design within a large industrial IT environment.

Conclusions:

• The systematic evaluation and integration approach (TAPE) is effective for industrial adoption of test-case generation tools.
• AI and algorithm-driven tools can enhance the efficiency and quality of software testing for REST APIs.
• The study confirms the value of these tools in streamlining IT product development and service delivery.