Establishing Reliability and Validity of an Online Placement Test in an Omani Higher Education Institution

Abstract

Although placing students in the appropriate proficiency levels of post-secondary English programs is crucial for optimal learning, the evaluation of placement tests (PTs) in terms of establishing their reliability and validity is relatively under-researched. This study assesses the validity, reliability, and effectiveness of an in-house online PT. The content validity was established through the internal and external moderation of the question papers and answer keys while criterion-related concurrent validity was established via IELTS benchmarking. New Student Survey was used to investigate the face validity. The internal consistency and reliability of the reading test items were measured using Cronbach’s alpha while descriptive statistics were calculated for the listening test. Paired sample t-test (dependent t-test) was used to assess the inter-rater reliability of the speaking and writing tests which were double-marked. The data analysis revealed that the PT was effective in placing students at different levels of the foundation program (FP) and the statistical analyses conducted to test the reliability and validity showed positive results for most of the test versions. The study offers useful insights to test developers and policymakers regarding the authentication of in-house tests and the creation of guidelines for PT design and evaluation.