Conducting Developmental Research Online vs. In-Person: A Meta-Analysis

Abstract

Abstract An increasing number of psychological experiments with children are being conducted using online platforms, in part due to the COVID-19 pandemic. Individual replications have compared the findings of particular experiments online and in-person, but the general effect of data collection method on data collected from children is still unknown. Therefore, the goal of the current meta-analysis is to estimate the average difference in effect size for developmental studies conducted online compared to the same studies conducted in-person. Our pre-registered analysis includes 211 effect sizes calculated from 30 papers with 3282 children, ranging in age from four months to six years. The estimated effect size for studies conducted online was slightly smaller than for their counterparts conducted in-person, a difference of d = −.05, but this difference was not significant, 95% CI = [−.17, .07]. We examined several potential moderators of the effect of online testing, including the role of dependent measure (looking vs verbal), online study method (moderated vs unmoderated), and age, but none of these were significant. The literature to date thus suggests—on average—small differences in results between in-person and online experimentation.