Full Program »
ENIDrift: A Fast and Adaptive Ensemble System for Network Intrusion Detection under Real-world Drift
Machine Learning (ML) techniques have been widely applied for network intrusion detection. However, existing ML-based network intrusion detection systems (NIDS) suffer from fundamental limitations that hinder them from being deployed in the real world. They consider a narrow scope rather than real-world drift that involves dynamically distributed network packets and well-crafted ML attacks. Besides, they pose high runtime overhead and have low processing speed.
In this paper, we solve the limitations and design ENIDrift, a fast and adaptive ensemble system for real-world network intrusion detection. ENIDrift employs iP2V, a novel incremental feature extraction method based on network packet fields, which adopts a simple three-layer neural network with relatively lightweight computation and achieves high efficiency. ENIDrift uses a robust sub-classifier generation module that constructs new sub-classifiers based on the stability and accuracy of incoming data chunks, and its training time is also reduced from O(n) to O(1). We extend the threat model and place experiments in real-world settings. We also collect and open-source our dataset, RWDIDS, which contains intense drifts for NIDS. Our extensive evaluation under real-world drift demonstrates that ENIDrift significantly outperforms the state-of-the-art solutions by up to 69.78% of F1 and reduces running time by 87.6%. ENIDrift also achieves a 100% F1 against our adversarial attack and is adaptive to various real-world drifts. Our field test also shows ENIDrift functions well even with delayed, inadequate training data, which is practical for real-world usage.