HMMED: A Multimodal Model with Separate Head and Payload Processing for Malicious Encrypted Traffic Detection

Abstract

Malicious encrypted traffic detection is a critical component of network security management. Previous detection methods can be categorized into two classes as follows: one is to use the feature engineering method to construct traffic features for classification and the other is to use the end-to-end method that directly inputs the original traffic to obtain traffic features for classification. Both of the abovementioned two methods have the problem that the obtained features cannot fully characterize the traffic. To this end, this paper proposes a hierarchical multimodal deep learning model (HMMED) for malicious encrypted traffic detection. This model adopts the abovementioned two feature generation methods to learn the features of payload and header, respectively, then fuses the features to get the final traffic features, and finally inputs the final traffic features into the softmax classifier for classification. In addition, since traditional deep learning is highly dependent on the training set size and data distribution, resulting in a model that is not very generalizable and difficult to adapt to unseen encrypted traffic, the model proposed in this paper uses a large amount of unlabeled encrypted traffic in the pretraining layer to pretrain a submodel used to obtain a generic packet payload representation. The test results on the USTC-TFC2016 dataset show that the proposed model can effectively solve the problem of insufficient feature extraction of traditional detection methods and improve the ACC of malicious encrypted traffic detection.