Examining the subjective fairness of at-home and online tests: Taking Duolingo English Test as an example

Abstract

The Duolingo English Test, a language proficiency test offered online, is now getting prevalent worldwide. A recap of existing literature denotes that there is an insufficient examination of the DET, particularly on its issues of fairness. Besides, empirical test fairness research had mainly focused on the objective aspect but may have overlooked the importance of its subjective aspect. Additionally, compared with in-person tests, the fairness research of at-home tests lags far behind. Therefore, the current study investigated the DET fairness from test takers’ perspectives. A DET Fairness Questionnaire based on (Kunnan’s AJ, 2004) Test Fairness Framework (comprising validity, absence of bias, access, administration, and social consequences) was developed. Data were collected from 1,012 Chinese university students and processed through descriptive and factor analyses. The descriptive analyses revealed that test takers perceived the DET to be fair overall. Specifically, they perceived that they had equal access to the test, but the test was invalid; the factor analyses showed that test takers’ perceptions of DET fairness (especially perceived validity and access) had a significant effect on their test performance. Such findings suggest that the subjective test fairness as an essential component could not be neglected in appraising an assessment as it influences test takers’ performance, and DET developers may strive harder to enhance the validity of DET to provide a fairer testing environment for test takers.