A Survey on TLS-Encrypted Malware Network Traffic Analysis Applicable to Security Operations Centers

Abstract

Recently, a majority of security operations centers (SOCs) have been facing a critical issue of increased adoption of transport layer security (TLS) encryption on the Internet, in network traffic analysis (NTA). To this end, in this survey article, we present existing research on NTA and related areas, primarily focusing on TLS-encrypted traffic to detect and classify malicious traffic with deployment scenarios for SOCs. Security experts in SOCs and researchers in academia can obtain useful information from our survey, as the main focus of our survey is NTA methods applicable to malware detection and family classification. Especially, we have discussed pros and cons of three main deployment models for encrypted NTA: TLS interception, inspection using cryptographic functions, and passive inspection without decryption. In addition, we have discussed the state-of-the-art methods in TLS-encrypted NTA for each component of a machine learning pipeline, typically used in the state-of-the-art methods.