Operationalizing open and restricted-access data—Formulating verifiable criteria for the openness of data sets mentioned in biomedical research articles

Abstract

Abstract Increasing the availability of research data sets is a goal of many stakeholders in science, and monitoring related practices requires definitions of the entity in question. There are several, largely overlapping, definitions for open data. However, they have so far not been translated into operationalizations that would allow us to detect, in a structured and reproducible way, whether, for a specific research article, the underlying data have been shared. Here, we propose a detailed set of criteria to enable such assessments, focusing on biomedical research. We have used these criteria to distribute performance-oriented funding at a large university hospital and to monitor data sharing practices in a dashboard. In addition to fully open data, we include separate criteria for data sets with restricted access, which we also reward. The criteria are partly inspired by the FAIR principles, particularly findability and accessibility, but do not map onto individual principles. The criteria attribute open data status in a binary fashion, both to individual data sets and, ultimately, articles with which they were shared. The criteria allow a verifiable assessment, based on automated and manual screening steps, which we have implemented and validated, as described elsewhere. Here, we focus conceptually on assessing the presence of shared data.