Assessing Foreign Language Narrative Writing Through Automated Writing Evaluation

Abstract

Automated writing evaluation (AWE) has become increasingly popular in the assessment of writing. The study in this chapter examines the extent to which EFL learners' overall narrative writing performance improves through the AWE feedback system (i.e., Pigai). Eighteen university participants were required to write one paragraph narratives on the web-based Pigai system every week over the course of a month. Findings show a significant improvement in overall scores between the first and last writing task. The analysis of lexical profile further shows a significant improvement in lexical richness, clause density, and paragraph length between the first and last narrative task. The study also reported that the primary error types that occurred in learner narrative writing were lexical, mechanical, and syntactic errors. Results of post-writing interviews also showed a positive attitude towards Pigai. Finally, a positive correlation was observed between automated Pigai scores and human rating scores, supporting the reliability of the AWE system.