Usability evaluation with tasks characterized by the information search process

Abstract

Purpose The purpose of this study is to present an approach to evaluating the usability of digital libraries in terms of knowledge states within the information search process (ISP) by investigating the connections among components with which users interact, perceived usability and task stages. The China National Knowledge Infrastructure (CNKI), the most popular digital library in China, was chosen to be studied. Design/methodology/approach With the research framework, measurements were generated for perceived usability in task stages for efficiency, effectiveness and satisfaction. Two usability experiments and usability tests were administered to 22 subjects in two groups. Three information retrieval tasks were designed according to the characteristics of knowledge states within the ISP for each experiment. The transaction logs were captured and the experimental results were recorded. The users completed a pre-search questionnaire and post-search questionnaire, and interviews were conducted. Findings The study showed that both interactions on components of the digital library and usability scores differed with sequential tasks characterized by knowledge states within the ISP; new usability problems relating to the CNKI were found based on the empirical results. Originality/value Many studies on the usability evaluation of digital libraries focused on the outcome rather than on the ISP. This study proposed an effective approach to usability evaluation based on users’ knowledge states within the ISP.