Safeguarding Online Communications using DistilRoBERTa for Detection of Terrorism and Offensive Chats

Abstract

People use social media for both good and distasteful purposes. When used with malicious intent, it raises significant concerns as it involves the use of offensive language and hate speech that promote terrorism and other negative behaviors. To create a safe, secure and pleasant environment, these communications must be closely monitored to prevent severe problems, associated risks and other pertinent issues. With the help of AI, specifically Large Language Models (LLM), we can quickly analyze text and speech to determine whether the communications promote the dangers identified here above not to mention other toxic elements. For this research, the LLM used is the DistilRoBERTa model from the Transformers library using Hugging Face. The DistilRoBERTa model was trained on datasets consisting of terrorism-related conversations, offensive-related conversations, and neutral conversations. These datasets were obtained from publicly available sources. The results of the experimentation show that the model achieved 99% accuracy, precision, recall, F1 score, and ROC curve. To improve the robustness of the model, it must be continuously fine-tuned to predict dynamic communication behavior since real conversations are inaccessible due to restrictions. A drag-and-drop interface is used to upload the files and get the categorical output, ensuring seamless and easy interaction.