Periodicity in User Engagement with a Search Engine and Its Application to Online Controlled Experiments

Abstract

Nowadays, billions of people use the Web in connection with their daily needs. A significant part of these needs are constituted by search tasks that are usually addressed by search engines. Thus, daily search needs result in regular user engagement with a search engine. User engagement with web services was studied in various aspects, but there appears to be little work devoted to its regularity and periodicity. In this article, we study periodicity of user engagement with a popular search engine through applying spectrum analysis to temporal sequences of different engagement metrics. First, we found periodicity patterns of user engagement and revealed classes of users whose periodicity patterns do not change over a long period of time. In addition, we give an exhaustive analysis of the stability and quality of identified clusters. Second, we used the spectrum series as key metrics to evaluate search quality. We found that the novel periodicity metrics outperform the state-of-the-art quality metrics both in terms of significance level ( p -value) and sensitivity to a large set of larges-scale A/B experiments conducted on real search engine users.