MABERT: Mask-Attention-Based BERT for Chinese Event Extraction

Abstract

Event extraction is an essential but challenging task in information extraction. This task has considerably benefited from pre-trained language models, such as BERT. However, when it comes to the trigger-word mismatch problem in languages without natural delimiters, existing methods ignore the complement of lexical information to BERT. In addition, the inherent multi-role noise problem could limit the performance of methods when one sentence contains multiple events. In this article, we propose a Mask-Attention-based BERT (MABERT) framework for Chinese event extraction to address the above problems. Firstly, in order to avoid trigger-word mismatch and integrate lexical features into BERT layers directly, a mask-attention-based transformer augmented with two mask matrices is devised to replace the original one in BERT. By the mask-attention-based transformer, the character sequence interacts with external lexical semantics sufficiently and keeps its structure information at the same time. Moreover, against the multi-role noise problem, we make use of event type information from representation and classification, two aspects to enrich entity features, where type markers and event-schema-based mask matrix are proposed. Experimental results on the widely used ACE2005 dataset show the effectiveness of our proposed MABERT on Chinese event extraction task compared with other state-of-the-art methods.