Homogeneity in Web Search Results

Abstract

Access to diverse perspectives nurtures an informed citizenry. Google and Bing have emerged as the duopoly that largely arbitrates which English-language documents are seen by web searchers. We present our empirical study over the search results produced by Google and Bing that shows a large overlap. Thus, citizens may not gain different perspectives by simultaneously probing them for the same query. Fortunately, our study also shows that by mining Twitter data, one can obtain search results that are quite distinct from those produced by Google, Bing, and Bing News. Additionally, the users found those results to be quite informative. We also present two novel tools we designed for this study. One uses tensor analysis to derive low-dimensional compact representation of search results and study their behavior over time. The other uses machine learning and quantifies the similarity of results between two search engines by framing it as a prediction problem. Although these tools have different underpinnings, the analytical results obtained using them corroborate each other, which reinforces the confidence one can place in them for finding meaningful insights from big data.