Data Stream Evolution Diagnosis Using Recursive Wavelet Density Estimators

Abstract

Data streams are a new class of data that is becoming pervasively important in a wide range of applications, ranging from sensor networks, environmental monitoring to finance. In this article, we propose a novel framework for the online diagnosis of evolution of multidimensional streaming data that incorporates Recursive Wavelet Density Estimators into the context of Velocity Density Estimation. In the proposed framework changes in streaming data are characterized by the use of local and global evolution coefficients . In addition, we propose for the analysis of changes in the correlation structure of the data a recursive implementation of the Pearson correlation coefficient using exponential discounting. Two visualization tools, namely temporal and spatial velocity profiles, are extended in the context of the proposed framework. These are the three main advantages of the proposed method over previous approaches: (1) the memory storage required is minimal and independent of any window size; (2) it has a significantly lower computational complexity; and (3) it makes possible the fast diagnosis of data evolution at all dimensions and at relevant combinations of dimensions with only one pass of the data. With the help of the four examples, we show the framework’s relevance in a change detection context and its potential capability for real world applications.