Identifying Mixture Components From Large-Scale Keystroke Log Data

Abstract

In a computer-based writing assessment, massive keystroke log data can provide real-time information on students’ writing behaviors during text production. This research aims to quantify the writing process from a cognitive standpoint. The hope is that the quantification may contribute to establish a writing profile for each student to represent a student’s learning status. Such profiles may contain richer information to influence the ongoing and future writing instruction. Educational Testing Service (ETS) administered the assessment and collected a large sample of student essays. The sample used in this study contains nearly 1,000 essays collected across 24 schools in 18 U.S. states. Using a mixture of lognormal models, the main findings show that the estimated parameters on pause data are meaningful and interpretable with low-to-high cognitive processes. These findings are also consistent across two writing genres. Moreover, the mixture model captures aspects of the writing process not examined otherwise: (1) for some students, the model comparison criterion favored the three-component model, whereas for other students, the criterion favored the four-component model; and (2) students with low human scores have a wide range of values on the mixing proportion parameter, whereas students with higher scores do not possess this pattern.