Performance Assessments With Microworlds and Their Difficulty

Abstract

The use of microworlds (MWs), or complex dynamic systems, in educational testing and personnel selection is hampered by systematic measurement errors because these new and innovative item formats are not adequately controlled for their difficulty. This empirical study introduces a way to operationalize an MW's difficulty and demonstrates the strong effects of variations in it. A rationale for predicting an MW's difficulty (number and quality of relationships between the same number of variables) is tested for difficulty's impact on assessed performance. As the number of relationships increases, control performance, acquired knowledge, and self-efficacy as moderating variables decrease. Analysis of knowledge types (qualitative and quantitative) shows that system difficulty affects quantitative knowledge more than it does qualitative knowledge.