Machine learning-based test smell detection

Area of Science:

• Software Engineering
• Software Quality Assurance
• Machine Learning Applications

Background:

• Test smells indicate suboptimal design choices in test cases, negatively impacting maintainability and effectiveness.
• Automated heuristic-based techniques exist for test smell detection but have limited performance and rely on tunable thresholds.

Purpose of the Study:

• To design and evaluate a novel machine learning (ML)-based approach for detecting four types of test smells.
• To compare the performance of ML models against state-of-the-art heuristic-based detection techniques.

Main Methods:

• Development of the largest manually-validated dataset of test smells for experimentation.
• Training and assessment of six machine learning models in within- and cross-project scenarios.
• Comparative analysis of ML-based detection against existing heuristic methods.

Main Results:

• The ML-based approach demonstrated significantly better performance than heuristic techniques.
• However, no ML model achieved an average F-Measure exceeding 51%, indicating limited detection accuracy.
• A qualitative investigation identified current challenges hindering effective test smell detection.

Conclusions:

• While ML offers improvements, current approaches struggle to achieve high accuracy in test smell detection.
• Addressing identified challenges is crucial for advancing the field of automated test smell detection.
• Future research should focus on overcoming these limitations to enhance software quality assurance.