Features extraction and fusion by attention mechanism for software defect prediction

Area of Science:

• Computer Science
• Software Engineering
• Artificial Intelligence

Background:

• Software defect prediction models typically use metrics, semantic, and network features.
• Complex software structures and limited samples hinder accurate defect prediction due to ineffective feature representation and extraction.
• Single features often fail to capture the complete software structure.

Purpose of the Study:

• To propose an efficient and accurate method for representing Abstract Syntax Trees (AST).
• To introduce the Multi-Features Attention (MFA) model for enhanced software defect prediction.
• To leverage deformable and self-attention mechanisms for feature extraction and fusion.

Main Methods:

• Developed a novel method for efficient and accurate Abstract Syntax Tree (AST) representation.
• Proposed the Multi-Features Attention (MFA) model incorporating deformable attention for feature extraction.
• Utilized self-attention mechanisms to fuse semantic and network features for comprehensive analysis.
• Conducted cross-version and cross-project experiments on 21 Java projects.

Main Results:

• The MFA model achieved average ACC, F1, and AUC scores of 0.7, 0.614, and 0.711 in cross-version experiments.
• In cross-project experiments, MFA attained average ACC, F1, and AUC of 0.687, 0.575, and 0.696.
• MFA demonstrated performance improvements of up to 41% compared to other models.
• Fusion of multiple features proved superior to using single features for prediction.

Conclusions:

• The MFA model effectively represents ASTs and fuses diverse software features, significantly enhancing defect prediction accuracy.
• The proposed attention-based mechanisms provide a robust approach for extracting and integrating complex software characteristics.
• MFA offers a promising advancement in software defect prediction, outperforming existing methods in both cross-version and cross-project scenarios.