Comparison of the effect of the generative model on the performance of deep neural networks and transformer in contextual social bot detection

Abstract

Abstract considering the important role of social networks and the affectability of people on different issues by these networks, the presence of various types of bots in them is a security risk for the networks. There are many types of bot detection techniques, but the structure of social bots is constantly changing and updating, which makes them run away from bot detection techniques, and finding a sustainable approach to this problem has become a new issue. In addition, in the real world, social bots, despite the important role they have, are far fewer than real users, and the available data set reflects this. In certain social media, we can collect a million real profiles with an easy crawl, but finding a bot account is not that easy. Facing this problem, some bot detection methods, including supervised/unsupervised machine learning methods and neural networks, do not work accurately or are dysfunctional. The proposed method in this paper is to detect Twitter bots based on the content of tweets, which means the text of each tweet, and not using the highly updatable profile information. In this regard, recommended applying algorithms proper to analyze text data, one of which is based on a deep neural network, att_BiLSTM, which is a two-directional LSTM with an attention mechanism, and BERT, which is a transformer. It was also shown that by using the attention layer in BiLSTM, the model's accuracy would be near to the accuracy of BERT's transformer, which is an algorithm based on context. Also, with the presence of the attention layer in att_BiLSTM, the number of LSTM units in BILSTM can be less, and the same accuracy as BILSTM with a larger number of LSTM units is achieved. To reduce the imbalance of data and improve the accuracy of the bot detection, samples have been increased in a set of bots with a special type of Generative Adversarial Networks algorithm called Seq-GAN, which is suitable for discrete and sequential data. Then the difference performance in deep neural networks and transformer showed after using generative model.