An explorative study on document type assignment of review articles in Web of Science, Scopus and journals’ websites-Reference-Cited by-同舟云学术

An explorative study on document type assignment of review articles in Web of Science, Scopus and journals’ websites

Published:2024-02-01 Issue:1 Volume:9 Page:11-36
ISSN:2543-683X
Container-title:Journal of Data and Information Science
language:en
Short-container-title:

Author:

Zhu Manman¹²³,Lu Xinyue¹²,Chen Fuyou¹,Yang Liying¹²,Shen Zhesi¹

Affiliation:

1. National Science Library, Chinese Academy of Sciences , Beijing , China

2. Department of Information Resource Management, School of Economics and Management, University of Chinese Academy of Sciences , Beijing , China

3. Institute of Science and Technology Information, Jiangsu University , Zhenjiang , China

Abstract

Abstract Purpose Accurately assigning the document type of review articles in citation index databases like Web of Science(WoS) and Scopus is important. This study aims to investigate the document type assignation of review articles in Web of Science, Scopus and Publisher’s websites on a large scale. Design/methodology/approach 27,616 papers from 160 journals from 10 review journal series indexed in SCI are analyzed. The document types of these papers labeled on journals’ websites, and assigned by WoS and Scopus are retrieved and compared to determine the assigning accuracy and identify the possible reasons for wrongly assigning. For the document type labeled on the website, we further differentiate them into explicit review and implicit review based on whether the website directly indicates it is a review or not. Findings Overall, WoS and Scopus performed similarly, with an average precision of about 99% and recall of about 80%. However, there were some differences between WoS and Scopus across different journal series and within the same journal series. The assigning accuracy of WoS and Scopus for implicit reviews dropped significantly, especially for Scopus. Research limitations The document types we used as the gold standard were based on the journal websites’ labeling which were not manually validated one by one. We only studied the labeling performance for review articles published during 2017-2018 in review journals. Whether this conclusion can be extended to review articles published in non-review journals and most current situation is not very clear. Practical implications This study provides a reference for the accuracy of document type assigning of review articles in WoS and Scopus, and the identified pattern for assigning implicit reviews may be helpful to better labeling on websites, WoS and Scopus. Originality/value This study investigated the assigning accuracy of document type of reviews and identified the some patterns of wrong assignments.

Publisher

Walter de Gruyter GmbH

Link

https://www.sciendo.com/pdf/10.2478/jdis-2024-0003

Reference22 articles.

1. Aksnes, D. W. (2003). Characteristics of highly cited papers, Research Evaluation, 12, 159-170. doi: 10.3152/147154403781776645.

2. Baeza-Yates, R., and Ribeiro-Neto, B. (1999). Modern Information Retrieval (Addison-Wesley Longman Publishing Co., Inc.: Boston, MAUnited States).

3. Blümel, C., and Schniedermann, A. (2020). Studying review articles in scientometrics and beyond: a research agenda, Scientometrics, 124, 711-728. doi: 10.1007/s11192-020-03431-7.

4. Colebunders, R., and Rousseau, R. (2013). On the Definition of a Review, and Does It Matter? ISSI 2013, 272-274.

5. Davis, J. J., and Goadrich, M. H. (2006). “The relationship between Precision-Recall and ROC curves.” In Proceedings of the 23rd International Conference on Machine Learning, 233-240.