Affiliation:
1. National Research University Higher School of Economics
Abstract
<p>This article discusses the problem of task comparability with the help of scenario-based tasks for metacognitive skills. Using the data of “«4С”» tool for measuring critical thinking (N=500), the comparability of two scenarios within an identical digital environment with one set of indicators was investigated. The main difference in the scenarios lies in the contextual characteristics. The measurement invariance analysis of the instrument using confirmatory factor analysis was conducted. The results show that even with the equivalent construct structure and tasks’ characteristics, the context of the scenario has an effect on the student`’s performance. The main differences in results were recorded for tasks involving interaction with the environment, where the test-taker created an object with elements. Tasks involving working with text in a digital environment can be considered comparable in case of elements content change. The possible reasons behind the observed differences in scenarios are discussed.</p>
Publisher
Moscow State University of Psychology and Education
Subject
Developmental and Educational Psychology,Education,Social Psychology,Psychology (miscellaneous)
Reference17 articles.
1. Gracheva D.A., Tarasova K.V. Podhody k razrabotke variantov zadanij scenarnogo tipa v ramkah metoda dokazatel'noj argumentacii [Approaches to the development of scenario-based task forms within the framework of evidence-centered design]. Otechestvennaja i zarubezhnaja pedagogika [Domestic and foreign pedagogy], 2022, no. 3(1), pp. 83–97. (In Russ.).
2. Uglanova I.L., Orel E.A., Brun I.V. Izmerenie kreativnosti i kriticheskogo myshlenija v nachal'noj shkole [Measuring creativity and critical thinking in primary school]. Psihologicheskij Zhurnal [Psychological Journal], 2020, no. 6(41), pp. 96–107. (In Russ.).
3. Buerger S. [et al.]. What makes the difference? The impact of item properties on mode effects in reading assessments. Studies in Educational Evaluation, 2019. Vol. 62, pp. 1–9. DOI:10.1016/j.stueduc.2019.04.005
4. Chen F.F. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural equation modeling: a multidisciplinary journal, 2007. Vol. 14, no. 3, pp. 464–504. DOI:10.1080/10705510701301834
5. Crisp V. Exploring features that affect the difficulty and functioning of science exam questions for those with reading difficulties. Irish Educational Studies, 2011. Vol. 30, no. 3, pp. 323–343.