The effect of face-to-face versus online learning on student performance in anatomy: an observational study using a causal inference approach

Abstract

Abstract Introduction This study aimed to estimate the causal effect of face-to-face learning on student performance in anatomy, compared to online learning, by analysing examination marks under a causal structure. Methods We specified a causal graph to indicate how the mode of learning affected student performance. We sampled purposively to obtain end-semester examination marks of undergraduate and postgraduate students who learned using face-to-face (pre-COVID, 2019) or online modes (post-COVID, 2020). The analysis was informed by the causal graph. Marks were compared using linear regression, and sensitivity analyses were conducted to assess if effects were robust to unmeasured confounding. Results On average, face-to-face learning improved student performance in the end-semester examination in undergraduate students (gain of mean 8.3%, 95% CI 3.3 to 13.4%; E-value 2.77, lower limit of 95% CI 1.80) but lowered performance in postgraduate students (loss of 8.1%, 95% CI 3.6 to 12.6%; E-value 2.89, lower limit of 95% CI 1.88), compared to online learning. Discussion Under the assumed causal graph, we found that compared to online learning, face-to-face learning improved student performance in the end-semester examination in undergraduate students, but worsened student performance in postgraduate students. These findings suggest that different modes of learning may suit different types of students. Importantly, this is the first attempt to estimate causal effects of the mode of learning on student performance under a causal structure. This approach makes our assumptions transparent, informs data analysis, and is recommended when using observational data to make causal inferences.