Affiliation:
1. Computer Science and Engineering Department, Carlos III University of Madrid, 28911 Leganés, Spain
2. Engineering Departament, Libre University, Barranquilla 08002, Colombia
Abstract
The lack of quality in scientific documents affects how documents can be retrieved depending on a user query. Existing search tools for scientific documentation usually retrieve a vast number of documents, of which only a small fraction proves relevant to the user’s query. However, these documents do not always appear at the top of the retrieval process output. This is mainly due to the substantial volume of continuously generated information, which complicates the search and access not properly considering all metadata and content. Regarding document content, the way in which the author structures it and the way the user formulates the query can lead to linguistic differences, potentially resulting in issues of ambiguity between the vocabulary employed by authors and users. In this context, our research aims to address the challenge of evaluating the machine-processing quality of scientific documentation and measure its influence on the processes of indexing and information retrieval. To achieve this objective, we propose a set of indicators and metrics for the construction of the evaluation model. This set of quality indicators have been grouped into three main areas based on the principles of Open Science: accessibility, content, and reproducibility. In this sense, quality is defined as the value that determines whether a document meets the requirements to be retrieved successfully. To prioritize the different indicators, a hierarchical analysis process (AHP) has been carried out with the participation of three referees, obtaining as a result a set of nine weighted indicators. Furthermore, a method to implement the quality model has been designed to support the automatic evaluation of quality and perform the indexing and retrieval process. The impact of quality in the retrieval process has been validated through a case study comprising 120 scientific documents from the field of the computer science discipline and 25 queries, obtaining as a result 21% high, 39% low, and 40% moderate quality.
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference53 articles.
1. Zhang, X., Li, X., Jiang, S., Li, X., and Xie, B. (2019, January 18–20). Evolution Analysis of Information Retrieval based on co-word network. Proceedings of the 2019 3rd International Conference on Electronic Information Technology and Computer Engineering (EITCE), Xiamen, China.
2. Fuzzy retrieval algorithm for film and television animation resource database based on deep neural network;Tan;J. Radiat. Res. Appl. Sci.,2023
3. Efficient and secure content-based image retrieval with deep neural networks in the mobile cloud computing;Wang;Comput. Secur.,2023
4. Transformer based contextual text representation framework for intelligent information retrieval;Bhopale;Expert Syst. Appl.,2023
5. Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., and Gurevych, I. (2021). BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models 2021. arXiv.