Consensus-based guidance for conducting and reporting multi-analyst studies

Author:

Aczel Balazs1ORCID,Szaszi Barnabas1ORCID,Nilsonne Gustav23ORCID,van den Akker Olmo R4,Albers Casper J5,van Assen Marcel ALM46,Bastiaansen Jojanneke A78ORCID,Benjamin Daniel910ORCID,Boehm Udo11ORCID,Botvinik-Nezer Rotem12ORCID,Bringmann Laura F5ORCID,Busch Niko A13ORCID,Caruyer Emmanuel14ORCID,Cataldo Andrea M1516ORCID,Cowan Nelson17ORCID,Delios Andrew18ORCID,van Dongen Noah NN11ORCID,Donkin Chris19,van Doorn Johnny B11ORCID,Dreber Anna2021ORCID,Dutilh Gilles22,Egan Gary F23ORCID,Gernsbacher Morton Ann24ORCID,Hoekstra Rink5ORCID,Hoffmann Sabine25ORCID,Holzmeister Felix21ORCID,Huber Juergen21ORCID,Johannesson Magnus20ORCID,Jonas Kai J26,Kindel Alexander T27,Kirchler Michael21ORCID,Kunkels Yoram K7,Lindsay D Stephen28,Mangin Jean-Francois2930ORCID,Matzke Dora31,Munafò Marcus R32,Newell Ben R19ORCID,Nosek Brian A3334,Poldrack Russell A35ORCID,van Ravenzwaaij Don5ORCID,Rieskamp Jörg36ORCID,Salganik Matthew J27,Sarafoglou Alexandra31,Schonberg Tom37ORCID,Schweinsberg Martin38ORCID,Shanks David39ORCID,Silberzahn Raphael40,Simons Daniel J41,Spellman Barbara A34,St-Jean Samuel4243ORCID,Starns Jeffrey J44,Uhlmann Eric Luis45,Wicherts Jelte4ORCID,Wagenmakers Eric-Jan31

Affiliation:

1. ELTE Eotvos Lorand University

2. Karolinska Institutet

3. Stockholm University

4. Tilburg University

5. University of Groningen

6. Utrecht University

7. University Medical Center Groningen, University of Groningen

8. Friesland Mental Health Care Services

9. University of California Los Angeles

10. National Bureau of Economic Research

11. University of Amsterdam

12. Dartmouth College

13. University of Münster

14. University of Rennes, CNRS, Inria and Inserm

15. McLean Hospital

16. Harvard Medical School

17. University of Missouri

18. National University of Singapore

19. University of New South Wales

20. Stockholm School of Economics

21. University of Innsbruck

22. University Hospital Basel

23. Monash University

24. University of Wisconsin-Madison

25. Ludwig-Maximilians-University

26. Maastricht University

27. Princeton University

28. University of Victoria

29. Université Paris-Saclay

30. Neurospin, CEA

31. Amsterdam University

32. University of Bristol

33. Center for Open Science

34. University of Virginia

35. Stanford University

36. University of Basel

37. Tel Aviv University

38. ESMT Berlin

39. University College London

40. University of Sussex

41. University of Illinois

42. University of Alberta

43. Lund University

44. University of Massachusetts Amherst

45. INSEAD

Abstract

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

Funder

Netherlands Organisation for Scientific Research

Natural Sciences and Engineering Research Council of Canada

Fonds de Recherche du Québec - Nature et Technologies

European Research Council

VIDI fellowship organisation

VENI fellowship grant

National Science Foundation

Weizmann Institute of Science

John Templeton Foundation, Templeton World Charity Foundation, Templeton Religion Trust, and Arnold Ventures

Institut Européen d'Administration des Affaires

Publisher

eLife Sciences Publications, Ltd

Subject

General Immunology and Microbiology,General Biochemistry, Genetics and Molecular Biology,General Medicine,General Neuroscience

Cited by 30 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. Toward a more credible assessment of the credibility of science by many-analyst studies;Proceedings of the National Academy of Sciences;2024-09-05

2. Prevalence, patterns and predictors of paranormal beliefs in The Netherlands: a several-analysts approach;Royal Society Open Science;2024-09

3. Heterogeneity in effect size estimates;Proceedings of the National Academy of Sciences;2024-07-30

4. What is the vibration of effects?;BMJ Evidence-Based Medicine;2024-07-12

5. Subjective evidence evaluation survey for many-analysts studies;Royal Society Open Science;2024-07

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3