Scholarly article retrieval from Web of Science, Scopus and Dimensions: A comparative analysis of retrieval quality

Abstract

Scholarly databases are now being increasingly used for search and retrieval of research articles in different subject areas. Several previous studies have shown that different databases vary in their coverage of publication sources, and therefore, one may expect that for a given query, they may retrieve different results. However, how do these databases compare in terms of relevance of the retrieved results is relatively unexplored. This study, therefore, attempts to bridge this research gap by carrying out a systematic study of retrieval relevance of the three scholarly databases – Web of Science, Scopus and Dimensions. Five selected queries are used for this purpose. The retrieved results from the three databases for the given queries are first analysed in terms of volume of retrieved records, language of retrieved records, etc. Thereafter, a user-based annotation scheme is used to assess and compare the relevance of retrieved results. The standard measure of normalised discounted cumulative gain (NDCG) and Spearman rank correlation coefficient (SRCC) is computed for the purpose. Results indicate that although the number of retrieved results for the same query differs significantly in the three databases, the databases differ only marginally in retrieval relevance, with Web of Science having a slight edge over other two.