Three approaches to measuring recall on the Web: a systematic review

Abstract

PurposeThe purpose of this paper is to present a review on the use of the recall metric for evaluating information retrieval systems, especially search engines.Design/methodology/approachThis paper investigates different researchers’ views about recall metrics.FindingsFive different definitions for recall were identified. For the first group, recall refers to completeness, but it does not specify where all the relevant documents are located. For the second group, recall refers to retrieving all the relevant documents from the collection. However, it seems that the term “collection” is ambiguous. For the third group (first approach), collection means the index of search engines and, for the fourth group (second approach), collection refers to the Web. For the fifth group (third approach), ranking of the retrieved documents should also be accounted for in calculating recall.Practical implicationsIt can be said that in the first, second and third approaches, the components of the retrieval algorithm, the retrieval algorithm and crawler, and the retrieval algorithm and crawler and ranker, respectively, are evaluated. To determine the effectiveness of search engines for the use of users, it is better to use the third approach in recall measurement.Originality/valueThe value of this paper is to collect, identify and analyse literature that is used in recall. In addition, different views of researchers about recall are identified.