An integrated efficient solution for computing frequent and top- k elements in data streams

Abstract

We propose an approximate integrated approach for solving both problems of finding the most popular k elements, and finding frequent elements in a data stream coming from a large domain. Our solution is space efficient and reports both frequent and top- k elements with tight guarantees on errors. For general data distributions, our top- k algorithm returns k elements that have roughly the highest frequencies; and it uses limited space for calculating frequent elements. For realistic Zipfian data, the space requirement of the proposed algorithm for solving the exact frequent elements problem decreases dramatically with the parameter of the distribution; and for top- k queries, the analysis ensures that only the top- k elements, in the correct order, are reported. The experiments, using real and synthetic data sets, show space reductions with hardly any loss in accuracy. Having proved the effectiveness of the proposed approach through both analysis and experiments, we extend it to be able to answer continuous queries about frequent and top- k elements. Although the problems of incremental reporting of frequent and top- k elements are useful in many applications, to the best of our knowledge, no solution has been proposed.