Author:
Haouari Fatima,Elsayed Tamer
Abstract
AbstractSeveral studies examined the leverage of the stance in conversational threads or news articles as a signal for rumor verification. However, none of these studies leveraged the stance of trusted authorities. In this work, we define the task of detecting the stance of authorities towards rumors in Twitter, i.e., whether a tweet from an authority supports the rumor, denies it, or neither. We believe the task is useful to augment the sources of evidence exploited by existing rumor verification models. We construct and release the first Authority STance towards Rumors (AuSTR) dataset, where evidence is retrieved from authority timelines in Arabic Twitter. The collection comprises 811 (rumor tweet, authority tweet) pairs relevant to 292 unique rumors. Due to the relatively limited size of our dataset, we explore the adequacy of existing Arabic datasets of stance towards claims in training BERT-based models for our task, and the effect of augmenting AuSTR with those datasets. Our experiments show that, despite its limited size, a model trained solely on AuSTR with a class-balanced focus loss exhibits a comparable performance to the best studied combination of existing datasets augmented with AuSTR, achieving a performance of 0.84 macro-F1 and 0.78 F1 on debunking tweets. The results indicate that AuSTR can be sufficient for our task without the need for augmenting it with existing stance datasets. Finally, we conduct a thorough failure analysis to gain insights for the future directions on the task.
Funder
Qatar National Research Fund
Qatar University
Publisher
Springer Science and Business Media LLC
Subject
Computer Science Applications,Human-Computer Interaction,Media Technology,Communication,Information Systems
Reference76 articles.
1. Abdelali A, Mubarak H, Samih Y, Hassan S, Darwish K (2021) Qadi: Arabic dialect identification in the wild. In: Proceedings of the sixth Arabic natural language processing workshop, pp 1–10
2. Abdul-Mageed M, Elmadany A, et al (2021) Arbert & marbert: Deep bidirectional transformers for Arabic. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th International joint conference on natural language processing (Vol 1: Long Papers), pp 7088–7105
3. Abu Farha I, Magdy W (2021) Benchmarking transformer-based language models for arabic sentiment and sarcasm detection. In: Proceedings of the sixth Arabic natural language processing workshop, pp 21–31. Association for computational linguistics, Kyiv, Ukraine (Virtual)
4. Albalawi RM, Jamal AT, Khadidos AO, Alhothali AM (2023) Multimodal Arabic rumors detection. IEEEIEEEIEEE Access 11:9716–9730
5. Alhindi T, Alabdulkarim A, Alshehri A, Abdul-Mageed M, Nakov P (2021) AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking. NLP4IF 2021, 57