Abstract:

A malicious network traffic detection method based on multi-level distributed ensemble classifier was proposed for the problem that the attack model was not trained accurately due to the lack of some samples of attack steps for detecting attack in the current network big data environment,as well as the deficiency of the existing ensemble classifier in the construction of multilevel classifier.The dataset was first preprocessed and aggregated into different clusters,then noise processing on each cluster was performed,and then a multi-level distributed ensemble classifier,MLDE,was built to detect network malicious traffic.In the MLDE ensemble framework the base classifier was used at the bottom,while the non-bottom different ensemble classifiers were used.The framework was simple to be built.In the framework,big data sets were concurrently processed,and the size of ensemble classifier was adjusted according to the size of data sets.The experimental results show that the AUC value can reach 0.999 when MLDE base users random forest was used in the first layer，bagging was used in the second layer and AdaBoost classifier was used in the third layer.

Key words: malicious network traffic, attack detection, attack phase, network flow clustering, ensemble classifier