How Do Product Attributes and Reviews Moderate the Impact of Recommender Systems Through Purchase Stages?

Abstract

We investigate the moderating effect of product attributes and review ratings on views, conversion|views (conversion conditional on views), and final conversion of a purchase-based collaborative filtering recommender system on an e-commerce site. We run a randomized field experiment on a top retailer with 184,375 users split into a recommender-treated group and a control group. We tag theory-driven attributes of 37,125 unique products via Amazon Mechanical Turk to augment the usual product data (e.g., review ratings, descriptions). By examining the recommender’s impact through different stages—awareness (views), salience (conversion|views), and final conversion—and across product types, we provide nuanced insights. The study confirms that the recommender increases views, conversion|views, and final conversion rates by 15.3%, 21.6%, and 7.5%, respectively, but this lift is moderated by product attributes and review ratings. We find that the lift on product views is greater for utilitarian products compared with hedonic products as well as for experience products compared with search products. In contrast, the lift on conversion|views rate is greater for hedonic products compared with utilitarian products. Furthermore, the lift on views rate is greater for products with higher average review ratings, which suggests that a recommender acts as a complement to review ratings, whereas the opposite is true for conversion|views, where recommender and review ratings are substitutes. Additionally, a recommender’s awareness lift is greater than its saliency impact. We discuss the potential mechanisms behind our results as well as their managerial implications. This paper was accepted by David Simchi-Levi, information systems.