BERT4NID: An intra and inter packet representation with pre-training transformers for IoT network intrusion detection

It's crucial for IoT network intrusion detection models to learn discriminative traffic representations from observable network traffic. Existing methods primarily rely on manually crafted features to represent network traffic. However, these manually constructed features depend heavily on the knowledge and experience of network experts and are difficult to generalize to diverse and constantly changing network environments. Therefore, leveraging a large amount of unlabeled raw network traffic to learn traffic representations from observable traffic patterns has become a research hotspot in the field of network intrusion detection. This paper proposes a new IoT network intrusion detection model called Bidirectional Encoder Representations from Transformer for Network Intrusion Detection (BERT4NID), which learns both intra-packet and inter-packet features by proposing two novel proxy tasks and pre-training on a large amount of unlabeled data to obtain traffic representations. The pre-trained model can be fine-tuned with a small amount of labeled data for downstream classification tasks and achieve excellent performance. With only 1% of labeled training data, the model's F1 scores on MedBIoT and Kitsune datasets are 93.46% and 92.85%.