Examining the generalizability of research findings from archival data
-
Published:2022-07-19
Issue:30
Volume:119
Page:
-
ISSN:0027-8424
-
Container-title:Proceedings of the National Academy of Sciences
-
language:en
-
Short-container-title:Proc. Natl. Acad. Sci. U.S.A.
Author:
Delios Andrew1, Clemente Elena Giulia2ORCID, Wu Tao3ORCID, Tan Hongbin4ORCID, Wang Yong5ORCID, Gordon Michael6, Viganola Domenico7, Chen Zhaowei1ORCID, Dreber Anna28ORCID, Johannesson Magnus2ORCID, Pfeiffer Thomas6ORCID, Uhlmann Eric Luis9, Abd Al-Aziz Ahmad M., Abraham Ajay T., Trojan Jais, Adamkovic Matus, Agadullina Elena, Ahn Jungsoo, Akinci Cinla, Akkas Handan, Albrecht David, Alzahawi Shilaan, Amaral-Baptista Marcio, Anand Rahul, Ang Kevin Francis U., Anseel Frederik, Aruta John Jamir Benzon R., Ashraf Mujeeba, Baker Bradley J., Bao Xueqi, Baskin Ernest, Bathula Hanoku, Bauman Christopher W., Bavolar Jozef, Bayraktar Secil, Beckman Stephanie E., Benjamin Aaron S., Brown Stephanie E. V., Buckley Jeffrey, Buitrago R. Ricardo E., Bution Jefferson L., Byrd Nick, Carrera Clara, Caruso Eugene M., Chen Minxia, Chen Lin, Cicerali Eyyub Ensari, Cohen Eric D., Crede Marcus, Cummins Jamie, Dahlander Linus, Daniels David P., Daskalo Lea Liat, Dawson Ian G. J., Day Martin V., Dietl Erik, Domurat Artur, Dsilva Jacinta, Plessis Christilene du, Dubrov Dmitrii I., Edris Sarah, Elbaek Christian T., Elsherif Mahmoud M., Evans Thomas R., Fellenz Martin R., Fiedler Susann, Firat Mustafa, Freitag Raquel, Furrer Rémy A., Gautam Richa, Gautam Dhruba Kumar, Gearin Brian, Gerschewski Stephan, Ghasemi Omid, Ghasemi Zohreh, Ghosh Anindya, Giani Cinzia, Goldberg Matthew H., Goswami Manisha, Graf-Vlachy Lorenz, Griffith Jennifer A., Grigoryev Dmitry, Gu Jingyang, H Rajeshwari, Hadida Allegre L., Hafenbrack Andrew C., Hafenbrädl Sebastian, Hammersley Jonathan J., Han Hyemin, Harman Jason L., Hartanto Andree, Henkel Alexander P., Ho Yen-Chen, Holding Benjamin C., Holzmeister Felix, Horobet Alexandra, Huang Tina S.-T., Huang Yiming, Huntsinger Jeffrey R., Idzikowska Katarzyna, Imada Hirotaka, Imran Rabia, Ingels Michael J., Jaeger Bastian, Janssen Steve M. J., Jia Fanli, Jiménez Alfredo, Jin Jason Lu, Johannes Niklas, Jolles Daniel, Jozefiakova Bibiana, Kačmár Pavol, Kalandadze Tamara, Kalimeri Kyriaki, Kang Polly, Kantorowicz Jaroslaw, Karadağ Didar, Karimi-Rouzbahani Hamid, Kee Daisy Mui Hung, Keller Lucas, Khan Haider A., Knutsson Mikael, Kombeiz Olga, Korniychuk Aleksey, Kowal Marta, Leder Johannes, Liang Liang Wenhao, Liew Taegyeong (Tae), Lin Fangwen, Liu Chengwei, Liu Bin, Longo Maria Cristina, Lovakov Andrey, Low Mei Peng, Lucas Gerardus J. M., Lukason Oliver, Ly Albert L., Ma Zhuoran, Mafael Alexander, Mahar Elizabeth A., Mahmoudkalayeh Soheil, Manheim David, Marcus Alfred, Marsh Melvin S., Martin Jolie M., Martinez Luis E., Martinoli Mario, Martončik Marcel, Masters-Waage Theodore C., Mata Rui, Mazloomi Hamid, McCarthy Randy J., Millroth Philip, Mishra Mahima, Mishra Supriti, Mohr Alexander, Moreau David, Myer Annalisa, Nadler Amos, Nair Sudhir, Nilsonne Gustav, Niszczota Paweł, O'Mahony Aoife, Oberhauser Marc, Obloj Tomasz, Orhan Mehmet A., Oswald Flora, Otterbring Tobias, Otto Philipp E., Padrón-Hernández Ivar, Pan Alan J., Paruzel-Czachura Mariola, Pfuhl Gerit, Pirrone Angelo, Porcher Simon, Protzko John, Prox Constantin, Qi Shelly, Rahal Rima-Maria, Rahman Md. Shahinoor, Reina Michelle L., Rentala Satyanarayana, Riaz Zahid, Ropovik Ivan, Röseler Lukas, Ross Robert M., Rotella Amanda, Roth Leopold H. O., Roulet Thomas J., Rubin Matthew M., Sammartino Andre, Sanchez Johann, Saville Adrian D., Schaerer Michael, Schleu Joyce Elena, Schmallenbach Leo, Schnabel Landon, Spüntrup Frederik Schulze, Schumpe Birga M., Senanayake Tony, Seri Raffaello, Sheng Feng, Snider Roary E., Song Di, Song Victoria, Starnawska Sylwia E., Stern Kai A., Stevens Samantha M., Strømland Eirik, Su Wunhong, Sun Hao, Sweeney Kevin P., Takamatsu Reina, Terskova Maria, Tey Kian Siong, Tierney Warren, Todorova Mariya M., Tolstoy Daniel, Torkkeli Lasse, Tybur Joshua M., Valderrey Francisco J., Vallina-Hernandez Ana Maria, Vasudevan Ranjith P., Rao Gudivada Venkat, Vernet Antoine, Vissak Tiia, Voss Hinrich, Wahle Thorsten, Wai Jonathan, Wakabayashi Lauren E.T., Wang Junnan, Wang Peng, Wang Yating, Warmenhoven Robert W., Wennberg Karl, Wernicke Georg, Woike Jan K., Wollbrant Conny E., Woodin Greg, Wright Joshua D., Xia Qiong, Xie Zhenzhen, Yoon Sangsuk, Yuan Wenlong, Yuan Lin, Yucel Meltem, Zheng Zhao, Zhou Haibo, Zogmaister Cristina, Zultan Ro'i,
Affiliation:
1. Department of Strategy and Policy, National University of Singapore, 119245 Singapore 2. Department of Economics, Stockholm School of Economics, Stockholm, 113 83 Sweden 3. School of Management and Economics and Shenzhen Finance Institute, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), Shenzhen 518000, China 4. Advanced Institute of Business, Tongji University, Shanghai 200092, China 5. School of Management, Xi'an Jiaotong University, Xi'an, Shaanxi 710049, China 6. New Zealand Institute for Advanced Study, Massey University, Auckland 0745, New Zealand 7. Global Indicators Department, Development Economics Vice Presidency, World Bank Group, Washington, DC 20433, USA 8. Department of Economics, University of Innsbruck, 6020 Innsbruck, Austria 9. Department of Organizational Behaviour, Institut Européen d'Administration des Affaires (INSEAD), Singapore 138676
Abstract
This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability—for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
Funder
Knut och Alice Wallenbergs Stiftelse Jan Wallanders och Tom Hedelius Stiftelse samt Tore Browaldhs Stiftelse Ministry of Education (Singapore) Tier 1 Grant National Science Foundation of China R&D Research Grant
Publisher
Proceedings of the National Academy of Sciences
Subject
Multidisciplinary
Cited by
22 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
|
|