Reduce, reuse, recycle: Introducing MetaPipeX, a framework for analyses of multi‐lab data

Abstract

AbstractMulti‐lab projects are large scale collaborations between participating data collection sites that gather empirical evidence and (usually) analyze that evidence using meta‐analyses. They are a valuable form of scientific collaboration, produce outstanding data sets and are a great resource for third‐party researchers. Their data may be reanalyzed and used in research synthesis. Their repositories and code could provide guidance to future projects of this kind. But, while multi‐labs are similar in their structure and aggregate their data using meta‐analyses, they deploy a variety of different solutions regarding the storage structure in the repositories, the way the (analysis) code is structured and the file‐formats they provide. Continuing this trend implies that anyone who wants to work with data from multiple of these projects, or combine their datasets, is faced with an ever‐increasing complexity. Some of that complexity could be avoided. Here, we introduce MetaPipeX, a standardized framework to harmonize, document and analyze multi‐lab data. It features a pipeline conceptualization of the analysis and documentation process, an R‐package that implements both and a Shiny App (https://www.apps.meta-rep.lmu.de/metapipex/) that allows users to explore and visualize these data sets. We introduce the framework by describing its components and applying it to a practical example. Engaging with this form of collaboration and integrating it further into research practice will certainly be beneficial to quantitative sciences and we hope the framework provides a structure and tools to reduce effort for anyone who creates, re‐uses, harmonizes or learns about multi‐lab replication projects.