Construct Under-representation and Construct Irrelevant Variances on IELTS Academic Writing Task 1: Is There Any Threat to Validity?

Abstract

The study reports on the validity of IELTS Academic Writing Task One (IAWTO) and compares and assesses the performance descriptors, i.e., coherence and cohesion, lexical resource and grammatical range, employed on IAWTO and IELTS Academic Writing Task Two (IAWTT). To these objectives, the data used were 53 participants' responses to graphic prompts driven by IELTS scoring rubrics, descriptive prompt, and retrospective, rather than concurrent, think-aloud protocols for detecting the cognitive validity of responses. The results showed that IAWTO input was degenerate and insufficient, rendering the construct under-represented, i.e., narrowing the construct. It was also found that IAWTO displayed to be in tune with cognitive difficulty of diagram analysis and the intelligence-based design of the process chart, rather than bar chart, being thus symmetrical with variances irrelevant to construct; this is argued to be biased to one group: Leading to under-performance of one group in marked contrast to over-performance of another group. Added to that, qualitative results established on instructors' protocols were suggestive of the dominance of performance descriptors on IAWTT rather than on IAWTO. The pedagogical implications of this study are further argued.