Detecting Covert Disruptive Behavior in Online Interaction by Analyzing Conversational Features and Norm Violations

Abstract

Disruptive behavior is a prevalent threat to constructive online engagement. Covert behaviors, such as trolling, are especially challenging to detect automatically, because they utilize deceptive strategies to manipulate conversation. We illustrate a novel approach to their detection: analyzing conversational structures instead of focusing only on messages in isolation. Building on conversation analysis, we demonstrate that (1) conversational actions and their norms provide concepts for a deeper understanding of covert disruption, and that (2) machine learning, natural language processing and structural analysis of conversation can complement message-level features to create models that surpass earlier approaches to trolling detection. Our models, developed for detecting overt (aggression) as well as covert (trolling) behaviors using prior studies’ message-level features and new conversational action features, achieved high accuracies (0.90 and 0.92, respectively). The findings offer a theoretically grounded approach to computationally analyzing social media interaction and novel methods for effectively detecting covert disruptive conversations online.