Does Help Help? An Empirical Analysis of Social Desirability Bias in Ratings

Abstract

Review-in-review (RIR) is a feature that allows viewers to generate positive or negative evaluations for primary quality evaluations of a product (e.g., ratings and reviews). This study reveals that it can cause social desirability bias in primary ratings: Reviewers who desire social recognition are driven to adjust their ratings (about 7.4% likelihood) to elicit more helpful responses and avoid unhelpful ones. This bias can be shown as distorted conformity to the prior rating distribution or extremity, depending on the RIR types. The model identifies how bias magnitude correlates with users’ social characteristics, thereby identifying vulnerable individuals. Platforms can incentivize less vulnerable users and remind susceptible ones to decrease the bias and can supplement rating conditional on the identified vulnerability extent (e.g., the distribution by the “independent” raters) to mitigate the bias’s impact on rating viewers. The simulation analysis compares the bias under different counterfactual RIR system designs, finding a composite RIR system (e.g., helpful and unhelpful RIRs) partially neutralizes the bias, obviating the need to remove all RIR features. The model further adapts to evaluate underexplored RIRs forms and can provide a “de-biased” metric while preserving individual ratings.