Tamil Offensive Language Detection: Supervised versus Unsupervised Learning Approaches

Abstract

Studies on natural language processing are mainly conducted in English, with very few exploring languages that are under-resourced, including the Dravidian languages. We present a novel work in detecting offensive language using a corpus collected from YouTube containing comments in Tamil. The study specifically aims to compare two machine learning approaches—namely, supervised and unsupervised—to detect offensive patterns in textual communications. In the first setup, offensive language detection models were developed using traditional machine learning algorithms such as Random Forest, Logistic Regression, Support Vector Machine, and AdaBoost, and assessed based on human labeling. Conversely, we used K -means ( K = 2) to cluster the unlabeled data before training the same set of machine learning algorithms to detect offensive communications. Performance scores indicate unsupervised clustering to be more effective than human labeling with ensemble classifiers achieving an impressive accuracy of 99.70% and 99.87% respectively for balanced and imbalanced datasets, hence showing that the unsupervised approach can be used effectively to detect offensive language in low-resourced languages.