Comparative evaluation of approaches & tools for effective security testing of Web applications

Area of Science:

• Software Engineering
• Cybersecurity
• Application Security Testing

Background:

• Static Application Security Testing (SAST) and Dynamic Application Security Testing (DAST) are widely accepted as essential for comprehensive web application security.
• However, their comparative effectiveness against specific risk categories (OWASP Top 10:2021, CWE Top 25:2023) and severity levels lacks empirical evidence.
• This gap leads to inefficient security testing, increasing costs and time, especially given the rise in preventable security incidents.

Purpose of the Study:

• To empirically evaluate the effectiveness of popular, free, and open-source SAST and DAST tools.
• To determine which tools and approaches are best suited for detecting specific vulnerabilities within the OWASP Top 10:2021 and CWE Top 25:2023 lists.
• To provide evidence-based recommendations for selecting security testing tools based on risk category and severity.

Main Methods:

• Seventy-five real-world web applications across technology, health, and education domains were tested.
• Four SAST and five DAST open-source tools were utilized to scan each application.
• Vulnerability detection effectiveness was measured by the count and severity of identified issues, mapped against OWASP Top 10:2021 and CWE Top 25:2023.

Main Results:

• DAST tools were preferred for four OWASP Top 10:2021 categories, while SAST tools were preferred for three; two categories were equally addressed.
• For CWE Top 25:2023, all approaches (SAST, DAST, combined) were equally effective, covering three categories each.
• Significant limitations were observed, with no tools detecting vulnerabilities in one OWASP and eight CWE categories; OWASP ZAP (DAST) excelled in specific categories, Yasca (SAST) in high-severity flaws, and WASP/Vega (DAST) in medium/low-severity issues.

Conclusions:

• DAST tools demonstrate superiority in detecting certain vulnerability types, while SAST tools are indispensable for identifying high-severity issues.
• The study highlights critical limitations in current popular open-source tools, indicating a need for improved detection capabilities.
• Findings offer practical, evidence-based guidance for selecting SAST and DAST tools, optimizing security testing efficiency and cost-effectiveness for organizations.