A transfer-aware, deployment-oriented evaluation framework for NetFlow-based intrusion detection systems (TAN-IDS).

Area of Science:

• Cybersecurity
• Machine Learning
• Network Security

Background:

• Machine learning-based Intrusion Detection Systems (IDS) demonstrate high accuracy on single datasets but falter in real-world deployments due to domain shift.
• Existing evaluations often overlook the impact of varying network environments and feature-space incompatibilities on IDS performance.

Purpose of the Study:

• To introduce TAN-IDS, a transfer-aware and deployment-oriented evaluation framework for NetFlow-based intrusion detection.
• To bridge the gap between laboratory benchmarking and practical deployment of IDS by addressing domain shift.
• To enable reliable attribution of performance degradation to domain shift rather than feature-space issues.

Main Methods:

• Developed TAN-IDS, a framework unifying heterogeneous NetFlow datasets into a compact 8-dimensional feature interface.
• Formalized deployment conditions into evaluation scenarios: in-dataset, cross-dataset transfer, mixed-domain training, and fine-tuning.
• Conducted extensive experiments using diverse machine learning models and neural architectures, including a Transformer-based model.

Main Results:

• High in-dataset accuracy does not guarantee cross-dataset robustness; model complexity alone is insufficient to mitigate domain shift.
• Domain-aware training strategies significantly improve generalization and robustness.
• Fine-tuning with only 5% labeled target-domain data recovered over 95% attack-class recall and F1-macro in several scenarios.

Conclusions:

• TAN-IDS provides a reproducible, deployment-centric evaluation framework for IDS.
• The framework highlights robustness limitations often missed by traditional benchmark-centric evaluations.
• Domain-aware training and targeted fine-tuning are crucial for effective real-world IDS deployment.