Shallow and Deep Syntactic/Semantic Structures for Passage Reranking in Question-Answering Systems

Abstract

In this article, we extensively study the use of syntactic and semantic structures obtained with shallow and full syntactic parsers for answer passage reranking. We propose several dependency and constituent-based structures, also enriched with Linked Open Data (LD) knowledge to represent pairs of questions and answer passages. We encode such tree structures in learning-to-rank (L2R) algorithms using tree kernels, which can project them in tree substructure spaces, where each dimension represents a powerful syntactic/semantic feature. Additionally, since we define links between question and passage structures, our tree kernel spaces also include relational structural features. We carried out an extensive comparative experimentation of our models for automatic answer selection benchmarks on different TREC QA corpora as well as the newer Wikipedia-based dataset, namely WikiQA, which has been widely used to test sentence rerankers. The results consistently demonstrate that our structural semantic models achieve the state of the art in passage reranking. In particular, we derived the following important findings: (i) relational syntactic structures are essential to achieve superior results; (ii) models trained with dependency trees can outperform those trained with shallow trees, e.g., in case of sentence reranking; (iii) external knowledge automatically generated with focus and question classifiers is very effective; and (iv) the semantic information derived by LD and incorporated in syntactic structures can be used to replace the knowledge provided by the above-mentioned classifiers. This is a remarkable advantage as it enables our models to increase coverage and portability over new domains.