Repetitive research: a conceptual space and terminology of replication, reproduction, revision, reanalysis, reinvestigation and reuse in digital humanities

Abstract

AbstractThis article is motivated by the ‘reproducibility crisis’ that is being discussed intensely in fields such as Psychology or Biology but is also becoming increasingly relevant to Artificial Intelligence, Natural Language Processing and Digital Humanities, not least in the context of Open Science. Using the phrase ‘repetitive research’ as an umbrella term for a range of practices from replication to follow-up research, and with the objective to provide clarity and help establish best practices in this area, this article focuses on two issues: First, the conceptual space of repetitive research is described across five key dimensions, namely those of the research question or hypothesis, the dataset, the method of analysis, the team, and the results or conclusions. Second, building on this new description of the conceptual space and on earlier terminological work, a specific set of terms for recurring scenarios of repetitive research is proposed. For each scenario, its position in the conceptual space is defined, its typical purpose and added value in the research process are discussed, the requirements for enabling it are described, and illustrative examples from the domain of Computational Literary Studies are provided. The key contribution of this article, therefore, is a proposal for a transparent terminology underpinned by a systematic model of the conceptual space of repetitive research.