GAMPAL: an anomaly detection mechanism for Internet backbone traffic by flow size prediction with LSTM-RNN

Abstract

AbstractThis paper proposes a general-purpose anomaly detection mechanism for Internet backbone traffic named GAMPAL (General-purpose Anomaly detection Mechanism using Prefix Aggregate without Labeled data). GAMPAL does not require labeled data to achieve general-purpose anomaly detection. For scalability to the number of entries in the BGP RIB (Border Gateway Protocol Routing Information Base), GAMPAL introduces prefix aggregate. The BGP RIB entries are classified into prefix aggregates, each of which is identified with the first three AS (Autonomous System) numbers in the AS_PATH attribute. GAMPAL establishes a prediction model for traffic sizes based on past traffic sizes. It adopts a LSTM-RNN (Long Short-Term Memory Recurrent Neural Network) model that focuses on the periodicity of the Internet traffic patterns at a weekly scale. The validity of GAMPAL is evaluated using real traffic information, BGP RIBs exported from the WIDE backbone network (AS2500), a nationwide backbone network for research and educational organizations in Japan, and the dataset of an ISP (Internet Service Provider) in Spain. As a result, GAMPAL successfully detects anomalies such as increased traffic due to an event, DDoS (Distributed Denial of Service) attacks targeted at a stub organization, a connection failure, an SSH (Secure Shell) scan attack, and anomaly spam.