A Novel Capsule Network with Attention Routing for Text Classification

Abstract

Convolutional Neural Networks(CNNs) and Recurrent Neural Networks (RNNs) often neglect the relationship between local and global semantics in text. In contrast, capsule networks encode word position information and multi-level semantic information using vector capsules and capture the relationship between local and global semantics through dynamic routing. However, capsule networks commonly neglect contextual information during capsule generation. Moreover, complex dynamic routing in capsule networks results in significant computational cost during training and evaluation. Therefore, we introduce AARCapsNet, a novel capsule network with attention routing for text classification. AARCapsNet incorporates two well-designed routings: self-attention routing and fast attention routing. Self-attention routing encodes contextual information into semantic capsules while suppressing noisy capsules. Fast attention routing adaptively learns the connection relationship between semantic capsules and class capsules, which offers a cost-effective alternative to intricate dynamic routing. Experiments on five benchmark datasets demonstrate that our proposed method achieves competitive performance.