A Multi-Layer Feature Fusion Model Based on Convolution and Attention Mechanisms for Text Classification
-
Published:2023-07-24
Issue:14
Volume:13
Page:8550
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Yang Hua1, Zhang Shuxiang1, Shen Hao2, Zhang Gexiang3ORCID, Deng Xingquan1, Xiong Jianglin1, Feng Li1, Wang Junxiong1, Zhang Haifeng1, Sheng Shenyang1
Affiliation:
1. School of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430040, China 2. Baijuncheng Technology Co., Ltd., Wuhan 434000, China 3. School of Automation, Chengdu University of Information Technology, Chengdu 610059, China
Abstract
Text classification is one of the fundamental tasks in natural language processing and is widely applied in various domains. CNN effectively utilizes local features, while the Attention mechanism performs well in capturing content-based global interactions. In this paper, we propose a multi-layer feature fusion text classification model called CAC, based on the Combination of CNN and Attention. The model adopts the idea of first extracting local features and then calculating global attention, while drawing inspiration from the interaction process between membranes in membrane computing to improve the performance of text classification. Specifically, the CAC model utilizes the local feature extraction capability of CNN to transform the original semantics into a multi-dimensional feature space. Then, global attention is computed in each respective feature space to capture global contextual information within the text. Finally, the locally extracted features and globally extracted features are fused for classification. Experimental results on various public datasets demonstrate that the CAC model, which combines CNN and Attention, outperforms models that solely rely on the Attention mechanism. In terms of accuracy and performance, the CAC model also exhibits significant improvements over other models based on CNN, RNN, and Attention.
Funder
School Enterprise Cooperation Project Hubei Provincial Teaching and Research Project Ministry of Education Industry-University Cooperation Collaborative Education Project
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference51 articles.
1. Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., and Brown, D. (2019). Text classification algorithms: A survey. Information, 10. 2. Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2016). Bag of tricks for efficient text classification. arXiv. 3. Liu, J., Wang, X., Tan, Y., Huang, L., and Wang, Y. (2022). An attention-based multi-representational fusion method for social-media-based text classification. Information, 13. 4. Tayal, M.A., Bajaj, V., Gore, A., Yadav, P., and Chouhan, V. (2023, January 26–28). Automatic domain classification of text using machine learning. Proceedings of the 2023 International Conference on Communication, Circuits, and Systems (IC3S), Odisha, India. 5. Combining text classification with machine learning;Nwade;LAUTECH J. Eng. Technol.,2023
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|