Strategy use in a spoken dialog system–delivered paired discussion task: A stimulated recall study

Abstract

With recent technological advances, researchers have begun to explore the potential use of spoken dialog systems (SDSs) for L2 oral communication assessment. While several studies support the feasibility of building these systems for various types of oral tasks, research on the construct validity of SDS-delivered tasks is still limited. Thus, this study examines the cognitive processes engaged by an SDS-delivered paired discussion task, focusing on strategic competence, an essential component of L2 oral communication ability. Thirty adult test-takers completed a paired discussion task with an SDS acting as an interlocutor and provided stimulated recalls about their strategy use in the task. Three trained raters independently evaluated the test-takers’ oral task responses using a holistic rating scale devised for the task. Findings revealed the use of six categories of construct-relevant strategies during task performance. While no statistically significant differences were found in the use of these categories between high- and low-ability test-takers, marked differences were observed in the use of individual strategies within the categories between the test-takers at the two levels. These findings provide insight into how test-takers at different ability levels cognitively interact with SDS-delivered paired discussion tasks and offer implications for the design and validation of such tasks.