Attention-based Stacked Bidirectional Long Short-term Memory Model for Word Sense Disambiguation

Abstract

Word sense disambiguation is a basic task in Natural Language Processing which aims to identify the most appropriate sense of ambiguous words in different contexts by applying algorithm models. In this work, we propose a model that uses a stacked bidirectional Long Short-Term Memory neural network and attention mechanism to determine the sense of ambiguous words. First, the stacked bidirectional Long Short-Term Memory is employed for deep embedding-based representation of sentences containing ambiguous words. Then, we utilize the self-attention mechanism to highlight the contextual features of ambiguous words, and then construct the overall semantic representation of sentences. Finally, the sentence semantic representation is applied to the multilayer perception classifier to generate the appropriate category of the ambiguous word sense items. This model is tested on the Semeval-2007 task-17: English lexical samples dataset and using examples of ambiguous words sourced from Oxford, Cambridge, and Collins dictionaries as extra test datasets. The effectiveness of the proposed approach is demonstrated via comparison with existing word sense disambiguation approaches. Our experimental results show that the proposed model outperforms other word sense disambiguation methods in terms of the evaluation metrics (Average Accuracy, Micro F1-Score, Kappa, and Matthews Correlation Coefficient), and exhibits strong interpretability.