A spam detection model based on the discriminative TF-IDF belief rule base

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Novel spam evolves rapidly, creating a scarcity of labeled data for early detection.
• Current spam detection models struggle with generalization and interpretability due to reliance on large datasets and high-dimensional features.
• Opacity in current models hinders error tracing and limits early threat response.

Purpose of the Study:

• To propose a novel belief rule base (BRB) spam detection model utilizing Discriminative TF-IDF (DTI-BRB).
• To address the challenges of small-sample conditions, poor generalization, and decision opacity in spam detection.
• To enhance early threat detection and response capabilities through an interpretable and accurate model.

Main Methods:

• Developed a Discriminative TF-IDF (TF-IDF) method to convert raw text into low-dimensional features, identifying terms indicative of spam or ham.
• Integrated the Discriminative TF-IDF features into a belief rule base (BRB) expert system to mitigate the combination explosion problem.
• Validated the DTI-BRB model through two case studies under small-sample conditions.

Main Results:

• The DTI-BRB model achieved high accuracies of 91.5% and 95.5% with only 200 samples in two distinct case studies.
• Demonstrated excellent predictive performance even with limited labeled data, outperforming traditional models in low-data scenarios.
• The rule-based reasoning of the BRB provided decision interpretability, facilitating error tracing.

Conclusions:

• The proposed DTI-BRB model effectively addresses the challenges of data scarcity and feature dimensionality in spam detection.
• The model offers a viable solution for early spam detection, providing both high accuracy and interpretable decisions.
• This approach enhances the practical application of expert systems in cybersecurity for evolving threats.