A relational extraction approach based on multiple embedding representations and multi-head self-attention

Abstract

Currently, word segmentation errors and polysemy problems are common in the field of Chinese relationship extraction. Although character-based model input can avoid word segmentation errors, in order to obtain the word information of a sentence, it is often necessary to introduce a dictionary or an external knowledge base to expand the word information, which requires a lot of manpower and time. In response to the above existing problems, this article uses characters as input, uses multiple embedding models to jointly form a character vector sequence, and obtains features containing character information through BiLSTM and attention layers; considering that convolutional neural networks are good at extracting local features, obtain features containing word information through multi-kernel convolutional layers and multi-head self-attention layers, and finally use a gating mechanism to fuse the features. The model was tested on the public SanWen data set and our own cultural-travel data set, and obtained F1 values of 61.22% and 60.26% respectively. Experimental results show that our method can achieve better relationship extraction effects without using word segmentation tools and without building a dictionary or external knowledge base, and the effect is better than most commonly used models currently.