Does Scientific Evaluation Matter? Improving Digital Simulation Games by Design-Based Research

Abstract

Grounded in a design-based research approach, the aim of this article is to determine whether scientific evaluations help to (a) identify and fix problems in educational interventions and (b) eventually foster a more effective and positive evaluated intervention. Therefore, data from a longer-term evaluation of short digital simulation games about the European Parliament for civic education in schools were used. The data included three cycles of interventions with pre- and post-evaluations starting with the first prototype in 2015/2016 (n = 209), the second cycle in 2017/18 (n = 97), and the last one in 2019/20 (n = 222). After each evaluation, major problems and critiques regarding the simulation game were discussed with the developers, and changes were implemented in the game design. The four most important problems, the processes by which they were improved and the reactions of the participants in the following evaluations are pointed out in the article. A comparison of the last and first evaluation cycle showed an overall improvement of the simulation game regarding its effectiveness in transferring EU knowledge and the participants’ general satisfaction with the simulation game. This study underlines the value of the design-based research approach for developing educational interventions and can be useful for further work on civic education measures and the implementation of digital simulation games.