Feature importance guided autoencoder for dimensionality reduction in intrusion detection systems

Intrusion detection systems (IDS) play a vital role in protecting computer networks from malicious activities. Dimensionality reduction techniques are commonly employed to enhance the effectiveness and accuracy of machine learning based IDS. In this study, we proposed an effective dimensionality reduction technique called feature importance-based autoencoder (FI-AE) for intrusion detection systems. Our proposed approach encompasses several key components. First, we introduce a novel feature importance method known as one-versus-all feature importance (OVA), which utilizes a random forest algorithm. Next, we train an autoencoder model using a weighted loss function that takes into account the feature importance values obtained through the OVA method. Finally, we utilized the trained autoencoder to reduce the number of features in the benchmark datasets, followed by the application of a random forest classifier to the reduced datasets. We tested our proposed model using three well-known datasets, namely NSL-KDD, UNSW-NB15, and CIC-IDS2017. The experiments revealed that the random forest classifier, combined with our proposed model, outperformed previous dimensionality reduction techniques in terms of accuracy and F1-score.