The Effects of Aggregated Search Coherence on Search Behavior

Abstract

Aggregated search is the task of combining results from multiple independent search systems in a single Search Engine Results Page (SERP). Aggregated search coherence refers to the extent to which different sources on the SERP focus on similar senses of an ambiguous or underspecified query. In previous studies, we found that the query senses in a set of vertical results can influence user engagement with the web results (the so-called “spillover” effect). In this work, we investigate five research questions (RQ1--RQ5) that extend our prior work. First, we investigate the extent to which results from different sources focus on different senses of an ambiguous query (RQ1). Second, we investigate how the vertical-to-web spillover effect varies across different verticals (RQ2). Then, we examine whether the level of spillover depends on the vertical position (RQ3) and on whether the vertical results are displayed with a border and different-colored background to distinguish them from the web results (RQ4). Finally, we propose a new method for displaying results from a particular vertical that are more consistent with the query senses in the web results (RQ5). We evaluate this new method based on how it influences users to make more correct decisions with respect to the web results—to engage with the web results when at least one of them is relevant and to avoid engaging with the web results otherwise. Our results show the following trends. In terms of RQ1, our analysis suggests that the top results from the web search engine are more diversified than the top results from our four different verticals considered (images, news, shopping, and video). In terms of RQ2, we found a stronger spillover effect for the images vertical than the news, shopping, and video verticals. In terms of RQ3, we found a stronger level of spillover when the vertical was positioned at the top of the SERP versus to the right side of the web results. In terms of RQ4, we found an interesting additive effect between the vertical’s position and displaying the vertical results enclosed in a border and with a different-colored background—the image vertical had no spillover when presented to the right side of the web results and with a border and background. Finally, in terms of RQ5, we found that our proposed vertical results selection approach can influence users to make more correct predictions about their level of engagement with the web results.