Finding more trustworthy answers: Various trustworthiness factors in question answering

Abstract

In the recent explosion of Web information, it is important to find not only appropriate, but also more trustworthy answers to user questions. This paper proposes an improved ranking model for question answering (QA) which is focused on various answer trustworthiness factors. Contrary to past research that simply focused on document quality, we have identified three different answer trustworthiness factors in multiple layers of answering processes: document quality, authority and reputation of answer sources, and appropriateness of answering method for a given question. Each of these factors is used in the answer selection as an input to the ranking scheme that can be tuned for the confidence value for a particular answer candidate. Through several experiments, we analysed the efficacy of our QA model from two points of view: indexing and answering. In indexing, distilling unreliable documents brings not only a 96% reduction in document size but also a 92% speed increase in indexing time. To reveal the effect of trustworthiness factors in answering, we conducted several experiments to determine the optimum combination of weights of sub-features for trustworthiness factors. Finally, the proposed method using all answer trustworthiness factors obtained an improvement in effectiveness over the simple routing QA by 150% in Top1. We also investigated improvement impacts according to answer trustworthiness factors.