Affiliation:
1. Software Engineering College, Zhengzhou University of Light Industry, China
2. School of Computer Science and Technology, Beijing Institute of Technology, China
3. College of Intelligence and Computing, Tianjin University, China
4. Artificial Intelligence Laboratory, China Mobile Communication Group Tianjin Co., Ltd., China
5. Centre for Smart Health, School of Nursing, Hong Kong Polytechnic University, China
Abstract
Sentiment and emotion, which correspond to long-term and short-lived human feelings, are closely linked to each other, leading to the fact that sentiment analysis and emotion recognition are also two interdependent tasks in natural language processing (NLP). One task often leverages the shared knowledge from another task and performs better when solved in a joint learning paradigm. Conversational context dependency, multi-modal interaction, and multi-task correlation are three key factors that contribute to this joint paradigm. However, none of the recent approaches have considered them in a unified framework. To fill this gap, we propose a multi-modal, multi-task interactive graph attention network, termed M3GAT, to simultaneously solve the three problems. At the heart of the model is a proposed interactive conversation graph layer containing three core sub-modules, which are: (1) local-global context connection for modeling both local and global conversational context, (2) cross-modal connection for learning multi-modal complementary and (3) cross-task connection for capturing the correlation across two tasks. Comprehensive experiments on three benchmarking datasets, MELD, MEISD, and MSED, show the effectiveness of M3GAT over state-of-the-art baselines with the margin of 1.88%, 5.37%, and 0.19% for sentiment analysis, and 1.99%, 3.65%, and 0.13% for emotion recognition, respectively. In addition, we also show the superiority of multi-task learning over the single-task framework.
Funder
The Hong Kong Polytechnic University
National Science Foundation of China
Novel Software Technology in Nanjing University
Industrial Science and Technology Research Project of Henan Province
Foundation of Key Laboratory of Dependable Service Computing in Cyber-Physical-Society (Ministry of Education), Chongqing University
Natural Science Foundation of Henan
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Science Applications,General Business, Management and Accounting,Information Systems
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Self-Adaptive Representation Learning Model for Multi-Modal Sentiment and Sarcasm Joint Analysis;ACM Transactions on Multimedia Computing, Communications, and Applications;2024-01-11
2. Moving From Narrative to Interactive Multi-Modal Sentiment Analysis: A Survey;ACM Transactions on Asian and Low-Resource Language Information Processing;2023-07-22