Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech

Abstract

Social media is a modern person’s digital voice to project and engage with new ideas and mobilise communities—a power shared with extremists. Given the societal risks of unvetted content-moderating algorithms forExtremism, Radicalisation,andHate speech(ERH)detection, responsible software engineering must understand the who, what, when, where, and why such models are necessary to protect user safetyandfree expression. Hence, we propose and examine the unique research field ofERH context miningto unify disjoint studies. Specifically, we evaluate the start-to-finish design process from socio-technical definition-building and dataset collection strategies to technical algorithm design and performance. Our 2015–2021 51-studySystematic Literature Review (SLR)provides the first cross-examination of textual, network, and visual approaches to detectingextremistaffiliation,hatefulcontent, andradicalisationtowards groups and movements. We identify consensus-driven ERH definitions and propose solutions to existing ideological and geographic biases, particularly due to the lack of research in Oceania/Australasia. Our hybridised investigation on Natural Language Processing, Community Detection, and visual-text models demonstrates the dominating performance of textual transformer-based algorithms. We conclude with vital recommendations for ERH context mining researchers and propose an uptake roadmap with guidelines for researchers, industries, and governments to enable a safer cyberspace.