Development of the automatic item generation system for the diagnosis of misconceptions about force and laws of motion

Abstract

The understanding of force and laws of motion is a fundamental foundation for learning mechanics and understanding other complex physics-related subjects. Automatic item generation (AIG) is also suitable for generating items and able to reduce the chance of item exposure. We, thus, developed an AIG system for the diagnosis of misconceptions about force and laws of motion in order to create a large number of quality items that would be used to diagnose students’ misconceptions. AIG system that has been developed contains 18 item models; it can generate 320-3,200 test items. The system contains six menus, i.e., (1) users’ data, (2) item models, (3) item generation, (4) test generation, (5) the users’ guide, and (6) the system’s developer. Based on the examination of AIG system’s quality by experts on educational assessment and experts on information technology, AIG’s quality in terms of utility, feasibility, propriety, and accuracy is at the highest level. The system was improved using the two dimensions of users’ experiences with physics instructors, i.e., (1) pragmatic dimension and (2) hedonic dimension. This research offers an approach to developing AIG system that responds to users’ needs.