Abstract
The task of question matching/retrieval focuses on determining whether two questions are semantically equivalent. It has garnered significant attention in the field of natural language processing (NLP) due to its commercial value. While neural network models have made great strides and achieved human-level accuracy, they still face challenges when handling complex scenarios. In this paper, we delve into the utilization of different specializations encoded in different layers of large-scale pre-trained language models (PTMs). We propose a novel attention-based model called ERNIE-ATT that effectively integrates the diverse levels of semantics acquired by PTMs, thereby enhancing robustness. Experimental evaluations on two challenging datasets showcase the superior performance of our proposed model. It outperforms not only traditional models that do not use PTMs but also exhibits a significant improvement over strong PTM-based models. These findings demonstrate the effectiveness of our approach in enhancing the robustness of question matching/retrieval systems.
Funder
Fundamental Research Funds for the Central Universities of South-Central Minzu University
Talent Introduction Program Funds for the Central Universities of South-Central Minzu University
Publisher
Public Library of Science (PLoS)
Reference28 articles.
1. Sriram B, Fuhry D, Demir E, Ferhatosmanoglu H, Demirbas M. Short text classification in twitter to improve information filtering. In: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval; 2010. p. 841–842.
2. Mohler M, Bunescu R, Mihalcea R. Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In: Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies; 2011. p. 752–762.
3. Zhu H, Chen Y, Yan J, Liu J, Hong Y, Chen Y, et al. DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models. In: Goldberg Y, Kozareva Z, Zhang Y, editors. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics; 2022. p. 7782–7794. Available from: https://aclanthology.org/2022.emnlp-main.531.
4. Pairwise contrastive learning for sentence semantic equivalence identification with limited supervision;T Shao;Knowledge-Based Systems,2023
5. Graph neural networks for natural language processing: A survey;L Wu;Foundations and Trends® in Machine Learning,2023