Debugging in Computational Thinking: A Meta-analysis on the Effects of Interventions on Debugging Skills

Abstract

Computational thinking (CT), an essential 21st century skill, incorporates key computer science concepts such as abstraction, algorithms, and debugging. Debugging is particularly underrepresented in the CT training literature. This multi-level meta-analysis focused on debugging as a core CT skill, and investigated the effects of various debugging interventions. Moderator analyses revealed which interventions were effective, in which situations, and for what kind of learner. A significant overall mean effect of debugging interventions ([Formula: see text] = 0.64, CI = (0.32, 0.96), p < .001), was found based on 62 effect sizes from 18 source articles. Significant between-studies variation indicated that true effects could range from −0.54 to 1.82. In addition, sensitivity analyses and checks on confounding provided further understandings of intervention features and their impacts. Interventions using enhanced debuggers and systematic instruction were particularly effective in fostering debugging skills. Debugging intervention effects varied by participant population and potentially by publication type. Moreover, debugging interventions had impact regardless of how debugging skills were measured, programming medium used, control-group type, and whether the study was randomized. Future studies should investigate the best practices for improving debugging abilities for whom and under what circumstances.