What’s in a Badge? A Computational Reproducibility Investigation of the Open Data Badge Policy in One Issue of <i>Psychological Science</i>-Reference-Cited by-同舟云学术

What’s in a Badge? A Computational Reproducibility Investigation of the Open Data Badge Policy in One Issue of Psychological Science

Published:2023-02-02 Issue:4 Volume:34 Page:512-522
ISSN:0956-7976
Container-title:Psychological Science
language:en
Short-container-title:Psychol Sci

Author:

Crüwell Sophia¹²^ORCID,Apthorp Deborah³⁴^ORCID,Baker Bradley J.⁵^ORCID,Colling Lincoln⁶^ORCID,Elson Malte⁷⁸^ORCID,Geiger Sandra J.⁹^ORCID,Lobentanzer Sebastian¹⁰,Monéger Jean¹¹¹²^ORCID,Patterson Alex¹³^ORCID,Schwarzkopf D. Samuel¹⁴¹⁵,Zaneva Mirela¹⁶^ORCID,Brown Nicholas J. L.¹⁷^ORCID

Affiliation:

1. Meta-Research Innovation Center Berlin (METRIC-B), QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Charité – Universitätsmedizin Berlin

2. Department of History and Philosophy of Science, University of Cambridge

3. School of Psychology, University of New England

4. School of Computing, Australian National University

5. Department of Sport and Recreation Management, Temple University

6. School of Psychology, University of Sussex

7. Faculty of Psychology, Ruhr University Bochum

8. Horst Görtz Institute for IT Security, Ruhr University Bochum

9. Environmental Psychology, Department of Cognition, Emotion, and Methods, Faculty of Psychology, University of Vienna

10. Institute for Computational Biomedicine, University Hospital Heidelberg, Germany

11. Department of Psychology, University of Poitiers

12. Research Center on Cognition and Learning, Centre National de la Recherche Scientifique (CNRS) 7295

13. Sheffield Methods Institute, The University of Sheffield

14. School of Optometry and Vision Science, University of Auckland

15. Experimental Psychology, University College London

16. Department of Experimental Psychology, University of Oxford

17. Department of Psychology, Linnaeus University

Abstract

In April 2019, Psychological Science published its first issue in which all Research Articles received the Open Data badge. We used that issue to investigate the effectiveness of this badge, focusing on the adherence to its aim at Psychological Science: sharing both data and code to ensure reproducibility of results. Twelve researchers of varying experience levels attempted to reproduce the results of the empirical articles in the target issue (at least three researchers per article). We found that all 14 articles provided at least some data and six provided analysis code, but only one article was rated to be exactly reproducible, and three were rated as essentially reproducible with minor deviations. We suggest that researchers should be encouraged to adhere to the higher standard in force at Psychological Science. Moreover, a check of reproducibility during peer review may be preferable to the disclosure method of awarding badges.

Funder

Economic and Social Research Council

Ministerium für Kultur und Wissenschaft des Landes Nordrhein-Westfalen

Publisher

SAGE Publications

Subject

General Psychology

Link

http://journals.sagepub.com/doi/pdf/10.1177/09567976221140828

Reference47 articles.

1. How to Automatically Document Data With the codebook Package to Facilitate Data Reuse

2. Reactivation of Previous Experiences in a Working Memory Task

3. Psychological Science Stepping Up a Level

4. Blohowiak B. B., Cohoon J., de-Wit L., Eich E., Farach F. J., Hasselman F., Holcombe A. O., Humphreys M., Lewis M., Nosek B. A., Peirce J., Spies J. R., Seto C., Bowman S., Green D., Nilsonne G., Grahe J., Wykstra S., Hofelich Mohr A. Lowrey O. (2022, February 4). Badges to acknowledge open practices. OSF. https://osf.io/tvyxz

5. Computational Reproducibility via Containers in Psychology

Cited by 17 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Tactics of just, equitable, diverse, and inclusive scientific research;Journal of the Experimental Analysis of Behavior;2024-08-19

2. Open Science at Clinical Psychological Science: Reflections on Progress, Lessons Learned, and Suggestions for Continued Improvement;Clinical Psychological Science;2024-08-08

3. Population neuroscience data sharing: Policy ecology, infrastructure, practices and challenges;Chinese Science Bulletin;2024-07-30

4. Open science practices in demographic research: An appraisal;Demographic Research;2024-06-07

5. Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data;PLOS ONE;2024-03-13