ReproSchema: Enhancing Research Reproducibility through Standardized Survey Data Collection (Preprint)

Abstract

Inconsistencies in survey-type assessments (e.g., questionnaires) data collection across biomedical, clinical, behavioral, and social sciences pose challenges to research reproducibility. ReproSchema offers a schema-centric framework and comprehensive tools to standardize survey (e.g., assessment) design and facilitate reproducible data collection in multiple scenarios.

This study illustrates ReproSchema’s impact on enhancing research reproducibility and reliability. We first introduce ReproSchema’s conceptual and practical foundations, then compare it against twelve platforms, assessing its contributions in resolving inconsistencies in data collection. Three use cases detail ReproSchema’s application in standardizing required mental health common data elements, tracking changes in longitudinal data collection, and creating interactive checklists for neuroimaging research.

We describe ReproSchema’s foundation and practical implementation before selecting twelve platforms for comparison, including CEDAR, former, Kobo Toolbox, LORIS, MindLogger, OpenClinica, Pavlovia.org, PsyToolkit, Qualtrics, REDCap, SurveyCTO, SurveyMonkey. Our comparison focuses on adapted FAIR principles (i.e., Findability, Accessibility, Interoperability, and Reusability) and survey-platform-generic functions (i.e., shared assessment, multilingual, multimedia, validation, branch logic, scoring logic, adaptability, and non-code). We then present three use cases of survey design—NIMH-Minimal, the Adolescent Brain Cognitive Development (ABCD) and HEALthy Brain and Child Development Study (HBCD), and the Committee on Best Practices in Data Analysis and Sharing Checklist (eCOBIDAS)—to demonstration ReproSchema’s versatile applications.

ReproSchema standardizes survey-based data collection through a central schema and other synergistic components (e.g., a library of assessments, a toolkit for format conversion and schema validation, a user interface for data collection, and a template for multi-assessment research protocol creation). In the platform comparisons, ReproSchema is one of the few platforms that meet all criteria related to the adapted FAIR principles and six out of eight functionalities. Additionally, three use cases highlight ReproSchema’s effectiveness in streamlining data collection, enforcing version control, and facilitating data harmonization post-collection.

ReproSchema contributes to reproducible data collection through the standardized creation and usage of assessments in diverse research settings while being equipped with the general functions of other survey platforms. ReproSchema’s existing limitations and plan enhancements, including ontology mappings and semantic search capabilities, demonstrate ongoing refinement in utility for the research community.