Ranking the causal impact of recommendations under collider bias in k -spots recommender systems

Abstract

The first objective of recommender systems is to provide personalized recommendations for each user. However, personalization may not be its only use. Past recommendations can be further analyzed to gain global insights into users’ behavior with respect to recommended items. Such insights can help to answer design-related questions such as which items’ recommendations are the most impactful in terms of users’ utility, which type of recommendations are the most followed ones, which items could be dropped from the catalog, or which recommendations are under-performing compared to what one would expect. In order to answer those questions, we need to rank item recommendations’ performances in terms of their causal impact on some user-related outcome measures. Unfortunately, in previous work leveraging causal inference for recommendation systems, the attention is fully focused on correcting confounding bias and not on the collider bias. This bias is particularly relevant in the recommender context, where multiple items are simultaneously recommended. Indeed, when there is a fixed number of available spots (i.e., k -spots) and recommendations need to be provided at each session, we argue that it is not possible to estimate the causal impacts of recommendations but only the differences between them. Therefore, in this paper, we provide an unbiased estimator of the differences in the impacts of items’ recommendations, that work for any outcome of interest, and any type of recommender system as long as it has some degree of randomization. We apply our results both in a simulated environment and in a real-world offline environment leveraging logged data for recommended items in a digital healthcare app.