Perceptions and Experiences of Severe Content in Content Moderation Teams: A Qualitative Study

Abstract

AbstractExisting Trust & Safety policies predominantly focus on protecting content moderators’ (CoMos) safety against severe content violations. Nevertheless, in reality, CoMos might encounter content across a broad severity spectrum while perceptions of content severity among CoMos are yet to be fully understood in relation to their well-being and job accuracy. The current study employs a sequential study design to qualitatively examine the views of 34 CoMos in the USA and the Philippines and of 166 professionals (i.e., mental health and learning experience staff, team leads) supporting CoMos on content severity and its perceived impact on CoMos well-being and job accuracy. Of note, consistent with ethical guidelines and wellness best practices, all CoMos received tiered wellness support services from TaskUs Inc. Collected data were thematically analyzed and two overarching themes emerged. Within the first theme, namely, ‘perceived content categories’, both CoMos and professionals, after considering the potential distress caused by various contents, broadly distinguished between ‘moderate’ and ‘graphic’ content, while they further attributed their own definitions to each of the two categories. Under theme two—‘content impact on perceived stress- most study participants discussed how reviewing moderate content might have a more detrimental impact upon CoMos stress and job accuracy compared to reviewing graphic content in the lack of wellness support and care. This study is the first of its kind to give voice to both CoMos and support staff to discuss in depth their views on content moderation while also making participant-led recommendations for work policy and future research.