The psychological impacts of content moderation on content moderators: A qualitative study

Abstract

Content moderation generally refers to the practice of monitoring user generated content and applying a set of rules defining what is acceptable. As a result, Content Moderators (CMs) are responsible for reviewing and removing harmful online material, which has the potential to cause psychological harm. However, little is known about the perceived impacts of the job. Semi-structured interviews were conducted with 11 commercial CMs who were exposed to child sexual abuse material (CSAM) as part of their job. Interviews were analysed using framework analysis to investigate any psychological impacts. Results demonstrated that CMs manifested with a range of symptoms consistent with experiencing repeated trauma. This included experiencing intrusive thoughts of CSAM, triggered by situations with similar contexts to those encountered at work, avoidance of children and negative cognitive and emotional effects such as cynicism, anxiety and detachment. These symptoms fit into a framework of post traumatic and secondary traumatic stress and suggest CMs wellbeing and exposure to stressful stimuli may be comparable to professionals working in the emergency services or caring professions, such as social workers. These personnel are at heightened risk of various problems including compassion fatigue, vicarious trauma and burnout. Companies which employ moderators should learn from these comparable professions and provide psychoeducation and trauma-informed care to CMs which have been demonstrated to help mitigate these challenges.