Unintended effects of algorithmic transparency: The mere prospect of an explanation can foster the illusion of understanding how an algorithm works

Abstract

AbstractThis research shows that merely believing that one can access an explanation of how an algorithm works can foster an illusory sense of understanding the algorithm, even when people do not actually access and read the explanation. This effect occurs because the belief that one can access an explanation provides a feeling of empowerment that fosters an illusory sense of understanding. In turn, this illusory sense of understanding can yield unfounded reliance on algorithmic determinations. We further show that this effect is moderated by the target of an explanation and by the perceived utility of an explanation in enabling consumers to use an algorithm more effectively. From a theoretical standpoint, we offer a novel psychological account of illusory understanding based on empowerment. From a practical standpoint, we point to an unintended effect of algorithmic transparency: merely knowing that one can access an explanation for how an algorithm works may lull consumers into a false sense of understanding that yields unfounded reliance on algorithmic recommendations.