A lightweight transformer based multi task learning model with dynamic weight allocation for improved vulnerability...

Area of Science:

• Computer Science
• Software Engineering
• Cybersecurity

Background:

• Accurate software vulnerability prediction is essential for mitigating security risks.
• Real-world datasets present challenges due to imbalance and complexity, hindering detection of rare vulnerabilities.
• Existing methods like single-task learning and ensemble approaches often fall short in these scenarios.

Purpose of the Study:

• To develop an advanced framework for improved vulnerability prediction.
• To enhance the detection of rare but critical software vulnerabilities.
• To address the limitations of traditional methods in handling imbalanced and complex datasets.

Main Methods:

• Proposed a novel Multi-Task Learning with Position Encoding and Lightweight Transformer (MTLPT) framework.
• Utilized custom lightweight Transformer blocks and position encoding for capturing code dependencies.
• Implemented a dynamic weight loss function to manage data imbalance.

Main Results:

• MTLPT demonstrated superior performance over traditional methods in key metrics (recall, F1-score, AUC, MCC).
• The model showed improved sensitivity in detecting rare vulnerabilities.
• Ablation studies confirmed the effectiveness of Transformer blocks, position encoding, and dynamic loss function.

Conclusions:

• The MTLPT framework offers a significant advancement in software vulnerability prediction.
• The proposed methods effectively handle complex and imbalanced datasets for more accurate risk identification.
• MTLPT enhances predictive accuracy and efficiency, crucial for proactive cybersecurity.