Author:
Ranasinghe Tharindu,Anuradha Isuri,Premasiri Damith,Silva Kanishka,Hettiarachchi Hansi,Uyangodage Lasitha,Zampieri Marcos
Abstract
AbstractThe widespread of offensive content online, such as hate speech and cyber-bullying, is a global phenomenon. This has sparked interest in the artificial intelligence (AI) and natural language processing (NLP) communities, motivating the development of various systems trained to detect potentially harmful content automatically. These systems require annotated datasets to train the machine learning (ML) models. However, with a few notable exceptions, most datasets on this topic have dealt with English and a few other high-resource languages. As a result, the research in offensive language identification has been limited to these languages. This paper addresses this gap by tackling offensive language identification in Sinhala, a low-resource Indo-Aryan language spoken by over 17 million people in Sri Lanka. We introduce the Sinhala Offensive Language Dataset (SOLD) and present multiple experiments on this dataset. SOLD is a manually annotated dataset containing 10,000 posts from Twitter annotated as offensive and not offensive at both sentence-level and token-level, improving the explainability of the ML models. SOLD is the first large publicly available offensive language dataset compiled for Sinhala. We also introduce SemiSOLD, a larger dataset containing more than 145,000 Sinhala tweets, annotated following a semi-supervised approach.
Publisher
Springer Science and Business Media LLC
Reference127 articles.
1. Abainia, K., Kara, K., & Hamouni, T. (2022). A new corpus and lexicon for offensive tamazight language detection. In 7th international workshop on social media world sensors. Sideways’22. Association for Computing Machinery. https://doi.org/10.1145/3544795.3544852
2. Alakrot, A., Murray, L., & Nikolov, N. S. (2018). Towards accurate detection of offensive language in online communication in arabic. Procedia Computer Science, 142, 315–320. https://doi.org/10.1016/j.procs.2018.10.491
3. Aroyehun, S. T., & Gelbukh, A. (2018). Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling. In Proceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018) (pp. 90–97). Association for Computational Linguistics. https://aclanthology.org/W18-4411
4. Assenmacher, D., Niemann, M., Müller, K., Seiler, M., Riehle, D. M., & Trautmann, H. (2021). Rp-mod & rp-crowd: Moderator- and crowd-annotated German news comment datasets. In Thirty-fifth conference on neural information processing systems datasets and benchmarks track (Round 2). https://openreview.net/forum?id=NfTU-wN8Uo
5. Bansal, T., Jha, R., & McCallum, A. (2020). Learning to few-shot learn across diverse natural language classification tasks. In Proceedings of the 28th international conference on computational linguistics (pp. 5108–5123). International Committee on Computational Linguistics. https://doi.org/10.18653/v1/2020.coling-main.448. https://aclanthology.org/2020.coling-main.448