Affiliation:
1. School of Computer and Information Engineering Hubei Normal University Huangshi China
2. Information Retrieval and Knowledge Management Research Lab, School of Information Technology York University Toronto Canada
3. Department of Computer Science Western University London Canada
Abstract
AbstractRecently, large pretrained language models (PLMs) have led a revolution in the information retrieval community. In most PLMs‐based retrieval frameworks, the ranking performance broadly depends on the model structure and the semantic complexity of the input text. Sequence‐to‐sequence generative models for question answering or text generation have proven to be competitive, so we wonder whether these models can improve ranking effectiveness by enhancing input semantics. This article introduces SE‐BERT, a semantically enhanced bidirectional encoder representation from transformers (BERT) based ranking framework that captures more semantic information by modifying the input text. SE‐BERT utilizes a pretrained generative language model to summarize both sides of the candidate passage and concatenate them into a new input sequence, allowing BERT to acquire more semantic information within the constraints of the input sequence's length. Experimental results from two Text Retrieval Conference datasets demonstrate that our approach's effectiveness increasing as the length of the input text increases.
Funder
China Scholarship Council
National Natural Science Foundation of China
Natural Sciences and Engineering Research Council of Canada
Subject
Artificial Intelligence,Computational Mathematics