A Synergistic Bidirectional LSTM and N-gram Multi-channel CNN Approach Based on BERT and FastText for Arabic Event Identification

Abstract

Event extraction from texts continues to pose a challenge for many NLP systems. This article presents a novel neural network architecture that can extract and classify events from Arabic sentences. The model combines word representations and Part-Of-Speech (POS) tags and uses a bidirectional LSTM layer and a dual combined convolutional neural network. The first layer of the network focuses on sentence representations, while the second layer focuses on POS representations. The model takes advantage of both N-gram character features from FastText and contextual representations from bidirectional encoder representations from transformers. This combination proves to be successful, as evidenced by the good results obtained from evaluating the model on the Arabic TimeML corpus. Our results show that combining both contextual and N-gram representations outperforms the traditional skip-gram model.