Test-based patch clustering for automatically-generated patches assessment

Area of Science:

• Software Engineering
• Automated Program Repair
• Software Testing

Background:

• Automated Program Repair (APR) techniques frequently suffer from overfitting, where generated patches appear correct but fail to fix underlying bugs or introduce new defects.
• This overfitting necessitates costly human validation of patches, hindering practical adoption of APR tools.
• Existing methods lack efficient ways to manage and assess the large number of plausible patches generated by APR tools.

Purpose of the Study:

• To introduce xTestCluster, a novel, lightweight, test-based patch clustering approach to reduce the burden of patch validation.
• To minimize the number of plausible patches requiring human review, thereby accelerating the identification of correct patches.
• To provide developers with enhanced information for patch assessment by leveraging dynamic patch behavior.

Main Methods:

• Developed xTestCluster, a test-based clustering approach that analyzes the dynamic behavior of software patches.
• Applied xTestCluster after the patch generation phase to cluster patches from multiple APR tools.
• Utilized newly generated test cases to group patches based on their failure patterns on these tests.

Main Results:

• xTestCluster successfully clustered 902 plausible patches from 21 Java APR tools.
• The approach achieved a median reduction of 50% in the number of patches requiring developer review.
• Generated new test cases that effectively exposed behavioral differences among generated patches.

Conclusions:

• xTestCluster significantly reduces the time and effort required for developers to validate APR-generated patches.
• The technique provides valuable new test cases that aid in distinguishing between correct and incorrect patches.
• xTestCluster complements existing patch assessment methods, improving the overall efficiency and reliability of Automated Program Repair.