Development and preliminary validation of an open access, open data and open outreach indicator

Abstract

We present the development and preliminary validation of a new person-centered indicator that we propose is named “OADO” after its target concepts: Open Access (OA), Open Data (OD) and Open Outreach (OO). The indicator is comprised of two factors: the research factor indicating the degree of OA articles and OD in research; and the communication factor indicating the degree of OO in communication activities in which a researcher has participated. We stipulate that the weighted version of this new indicator, the Weighted-OADO, can be used to assess the openness of researchers in relation to their peers from their own discipline, department, or even group/center. The OADO is developed and customized to the needs of Elsevier's Research Information Management System (RIMS) environment, Pure. This offers the advantage of more accurate interpretations and recommendations for action, as well as the possibility to be implemented (and further validated) by multiple institutions, allowing disciplinary comparisons of the open practices across multiple institutes. Therefore, the OADO provides recommendations for action, and enables institutes to make informed decisions based on the indicator's outcome. To test the validity of the OADO, we retrieved the Pure publication records from two departments for each of the five faculties of the University of Southern Denmark and calculated the OADO of 995 researchers in total. We checked for definition validity, actionability, transferability, possibility of unexpected discontinuities of the indicator, factor independence, normality of the indicator's distributions across the departments, and indicator reliability. Our findings reveal that the OADO is a reliable indicator for departments with normally distributed values with regards to their Weighted-OADO. Unfortunately, only two departments displayed normal distributions, one from the health sciences and one from engineering. For departments where the normality assumption is not satisfied, the OADO can still be useful as it can indicate the need for making a greater effort toward openness, and/or act as an incentive for detailed registration of research outputs and datasets.