Orthogonal projection for anomaly detection in networking datasets

Abstract

AbstractIn recent years, the impressive growth of new wireless technologies, together with the appearance of new requirements in applications and services, is progressively changing the use of networks. Due to the high mobility required, the network must adapt to the infrastructure to meet the demands of the users. As a result, service providers currently have to over-provision network capacity, which is costly. In addition, considering efficient resource planning in advance involves a lot of labor-intensive efforts. Consequently, network usage analysis is a very useful tool that allows network administrators to find patterns and anomalies. Whilst pattern detection provides administrators the ability to define the infrastructure, anomaly detection provides rich and valuable information for certain applications, for example, to avoid network saturation in urban areas during peak hours. This article proposes a new methodology based on orthogonal projections over Call Detail Records (CDR) for anomaly detection to help in the dynamic management of the network in an urban area. The method is evaluated in a real scenario provided by an Italian telecommunications operator, considering different locations in the Milan metropolitan area, differentiated by the geographic resolution of the data, reaching F1 scores above 0.8. In addition, a new ground truth is presented, hoping it will become a reference data set for the community, in the form of a set of locations that have been corroborated for use in evaluating anomaly detection techniques.